Skip to content

Hua work dbms#48

Merged
zaihuaji merged 5 commits intomainfrom
hua-work-dbms
Mar 16, 2026
Merged

Hua work dbms#48
zaihuaji merged 5 commits intomainfrom
hua-work-dbms

Conversation

@zaihuaji
Copy link
Collaborator

No description provided.

zaihuaji and others added 2 commits March 13, 2026 09:36
Merge pull request #46 from NCAR/hua-work-dbms
Copilot AI review requested due to automatic review settings March 16, 2026 14:26
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the pgchksum workflow for web-file checksum evaluation and publishes a new package version.

Changes:

  • Skip missing wfile records when expanding wid/dsid results into full wfile records, and adjust the effective count.
  • Switch the object-store bucket used for web-file object checks from rda-data to gdex-data.
  • Bump package version to 2.0.7.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/rda_python_dbms/pgchksum.py Filters out missing per-wid fetches and changes the object-store bucket used for web object checks.
pyproject.toml Increments project version.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

zaihuaji and others added 3 commits March 16, 2026 09:33
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates pgchksum’s object-store targeting to use new bucket names and reduces hardcoded bucket usage during checksum evaluation, alongside a package version bump.

Changes:

  • Switch web/saved object store buckets from rda-* to gdex-*.
  • Pass the bucket via self.PVALS['BUCKET'] into object-store checksum evaluation instead of hardcoding.
  • Make multi-dataset webfile collection skip missing pgget_wfile() results and compact the resulting record set.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/rda_python_dbms/pgchksum.py Updates bucket names, removes hardcoded bucket args during evaluation, and filters missing webfile records when aggregating.
pyproject.toml Bumps project version to 2.0.7.
Comments suppressed due to low confidence (2)

src/rda_python_dbms/pgchksum.py:185

  • When running with the -f option (self.PGSUM['f'] is non-empty), this method returns cnt but never updates self.PGSUM['c']. Since start_actions() uses fcnt = self.PGSUM['c'] to drive the evaluation loop, the default 0 will cause no files to be checked even though cnt > 0. Set self.PGSUM['c'] = cnt (or return the records and have start_actions() use the returned count) in the -f branch so the requested files actually get processed.
      if self.PGSUM['f']:
         pgrecs = self.pgmhget('wfile', 'wid, dsid', {'wfile' : self.PGSUM['f']})
         cnt = len(pgrecs['wid']) if pgrecs else 0
         if cnt > 0:
            self.PGSUM['f'] = {}

src/rda_python_dbms/pgchksum.py:240

  • Same issue as get_checksum_wfilelist(): in the -f path (cnt > 0 from self.PGSUM['f']), the function returns the number of matched records but does not update self.PGSUM['c']. start_actions() then iterates range(self.PGSUM['c']) and will skip evaluating the explicitly provided files when -c is not also specified. Update self.PGSUM['c'] to the returned count in this branch.
      fcnt = self.PGSUM['c']
      cnt = len(self.PGSUM['f'])
      if cnt > 0:
         self.PGSUM['f'] = self.pgmhget('sfile', flds, {'sfile' : self.PGSUM['f']})
         return (len(self.PGSUM['f']['sid']) if self.PGSUM['f'] else 0)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@zaihuaji zaihuaji merged commit c8aa7e6 into main Mar 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants