Query Tool result export enhancements (JSON/XML, encoding, BOM, copy-with-headers)#10062
Open
dpage wants to merge 1 commit into
Open
Query Tool result export enhancements (JSON/XML, encoding, BOM, copy-with-headers)#10062dpage wants to merge 1 commit into
dpage wants to merge 1 commit into
Conversation
Several long-standing requests around exporting/copying Query Tool results, all in the results download/copy path: - Save results as JSON or XML in addition to CSV, selectable from a drop-down on the "Save results to file" toolbar button. The download generator is now format-aware and streams JSON/XML as well as CSV. - New "Output file encoding" preference (CSV/TXT output) controlling the character encoding of saved results; defaults to utf-8. - New "Add byte order mark (BOM)?" preference that prepends a UTF BOM to saved CSV/TXT files for better interoperability with applications such as Microsoft Excel. - New "Copy with headers?" preference seeding the default state of the results grid "Copy with headers" toggle. Adds integration tests for the JSON/XML/BOM/encoding download paths and updates the preferences and Query Tool toolbar documentation. Closes pgadmin-org#3205 Closes pgadmin-org#4128 Closes pgadmin-org#4129 Closes pgadmin-org#6695
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (9)
👮 Files not reviewed due to content moderation or server errors (4)
WalkthroughQuery Tool result export is extended to support JSON and XML formats, configurable output encoding (UTF-8, UTF-16, Windows-1252, etc.), and optional UTF byte order marks. A new preference controls whether column headers are included by default when copying results grid data. ChangesQuery Tool Export Enhancements
Sequence DiagramsequenceDiagram
participant User
participant ResultSetToolbar
participant ResultSet
participant DownloadEndpoint
participant DatabaseDriver
User->>ResultSetToolbar: Click "Save as JSON/XML/CSV"
ResultSetToolbar->>ResultSetToolbar: downloadResult(format)
ResultSetToolbar->>ResultSet: TRIGGER_SAVE_RESULTS(dataFormat)
ResultSet->>ResultSet: saveResultsToFile(fileName, progress, dataFormat)
ResultSet->>DownloadEndpoint: POST with format parameter
DownloadEndpoint->>DatabaseDriver: execute_on_server_as_csv(data_format)
DatabaseDriver->>DatabaseDriver: Route to _generate_json/_generate_xml/CSV writer
DatabaseDriver-->>DownloadEndpoint: Streamed result chunks
DownloadEndpoint->>DownloadEndpoint: Apply encoding/BOM rules
DownloadEndpoint->>DownloadEndpoint: Set MIME type & filename extension
DownloadEndpoint-->>User: Download response (JSON/XML/CSV)
Estimated Code Review Effort🎯 3 (Moderate) | ⏱️ ~25 minutes Suggested Reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
@dpage You appear to be, as they say, on a roll. |
There was a problem hiding this comment.
Pull request overview
This PR enhances the pgAdmin Query Tool “save/copy results” path by adding JSON/XML exports, configurable output encoding + optional BOM for CSV/TXT exports, and a preference-seeded “copy with headers” default.
Changes:
- Add streaming JSON and XML export formats for Query Tool results (alongside existing CSV/TXT).
- Add Query Tool preferences for output file encoding, optional BOM, and default “copy with headers”.
- Extend integration tests and update documentation/release notes for the new export/copy behavior.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| web/pgadmin/utils/driver/psycopg3/connection.py | Add JSON/XML streaming generators and route export generation by format. |
| web/pgadmin/tools/sqleditor/utils/query_tool_preferences.py | Register new Query Tool preferences for encoding/BOM and copy-with-headers default. |
| web/pgadmin/tools/sqleditor/tests/test_download_csv_query_tool.py | Add integration scenarios covering JSON/XML export + encoding/BOM paths. |
| web/pgadmin/tools/sqleditor/static/js/components/sections/ResultSetToolbar.jsx | Add “Save results” split-button drop-down and seed copy-with-headers from preference. |
| web/pgadmin/tools/sqleditor/static/js/components/sections/ResultSet.jsx | Send requested export format to backend; map format to MIME type and file extension. |
| web/pgadmin/tools/sqleditor/init.py | Make download endpoint format-aware; apply encoding/BOM for CSV and UTF-8 for JSON/XML. |
| docs/en_US/release_notes_9_16.rst | Add release note entries for the new export/copy features. |
| docs/en_US/query_tool_toolbar.rst | Document the new export format drop-down and encoding/BOM settings. |
| docs/en_US/preferences.rst | Document new CSV/TXT Output and Results Grid preferences. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+2182
to
+2200
| is_utf = output_encoding.lower().replace('-', '').replace( | ||
| '_', '').startswith('utf') | ||
|
|
||
| str_gen = gen(conn_obj, | ||
| trans_obj, | ||
| quote=blueprint.csv_quoting.get(), | ||
| quote_char=blueprint.csv_quote_char.get(), | ||
| field_separator=blueprint.csv_field_separator.get(), | ||
| replace_nulls_with=blueprint.replace_nulls_with.get(), | ||
| data_format=data_format) | ||
|
|
||
| def encoded_gen(text_gen): | ||
| is_first_chunk = True | ||
| for chunk in text_gen: | ||
| if is_first_chunk: | ||
| is_first_chunk = False | ||
| if add_bom and is_utf: | ||
| chunk = '\ufeff' + chunk | ||
| yield chunk.encode(output_encoding, errors='replace') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A batch of long-standing Query Tool result export/copy enhancements, all in
the results download/copy path.
drop-down on the Save results to file toolbar button. The download
generator is now format-aware and streams JSON/XML as well as CSV. JSON/XML
are always emitted as UTF-8; XML emits column names as escaped
nameattributes so column names that are not valid XML element names are handled
safely.
the character encoding used when saving results; defaults to utf-8, with a
free-text option for encodings that are not listed.
CSV/TXT files for better interoperability with applications such as Microsoft
Excel. (Applies to CSV/TXT output only.)
grid "Copy with headers" toggle (still toggleable per-copy).
Testing
download paths through the real
/query_tool/download/endpoint; theexisting CSV scenarios continue to pass, confirming the generator refactor
is non-regressive.
pycodestyleandeslintclean.entries.
Closes #3205
Closes #4128
Closes #4129
Closes #6695
Summary by CodeRabbit
Release Notes
New Features
Documentation
Tests