Refactor GeoTIFF Phase 5e: extract _overview.py from _writer.py#2261
Merged
Conversation
Move the overview pyramid helpers out of ``_writer.py`` into a new ``_overview.py`` module: - ``_make_overview`` - ``_block_reduce_2d`` - ``_replicate_pad_2d`` - ``_resolve_int_nodata`` - ``_validate_overview_levels`` - ``OVERVIEW_METHODS`` and ``_MAX_OVERVIEW_LEVELS`` constants ``_writer.py`` re-exports the names so the public import path (``xrspatial.geotiff._writer``) keeps working for tests, the ``_writers`` subpackage, and ``_gpu_decode``. Internal importers in ``_writers/gpu.py`` and ``_gpu_decode.py`` are switched to the new ``_overview`` module directly. Behavior-neutral. ``_writer.py`` drops from 2238 to 1804 lines. Part of #2211.
brendancol
commented
May 21, 2026
Contributor
Author
brendancol
left a comment
There was a problem hiding this comment.
PR Review: Refactor GeoTIFF Phase 5e: extract _overview.py from _writer.py
Mechanical extraction. I verified the moved code matches origin/main byte-for-byte (modulo the new module docstring on OVERVIEW_METHODS) and that all import paths still resolve.
Blockers
None.
Suggestions
None.
Nits
xrspatial/geotiff/_writer.py:370-- the newfrom ._overview import (...)re-export block sits mid-file (after the compression helpers, before the strip writer). The sibling_write_layoutre-export from PR-G sits near the top imports at line 89. Co-locating the two re-export blocks would make the "this name lives elsewhere now" pattern easier to scan. Not worth a churn-only follow-up if PR-L is about to add a third re-export block in roughly the same neighbourhood.xrspatial/geotiff/_overview.py:22-23-- theOVERVIEW_METHODSconstant gained a one-line#:Sphinx docstring that the original in_writer.py:352did not have. Minor, behavior-neutral, but technically a non-mechanical edit in a mechanical-extraction PR. Fine to keep.
What looks good
- The five helpers and two constants land in
_overview.pywith their docstrings, comments, and issue references intact (#1613,#1623,#1766,#1975,#2105). _PARALLEL_MIN_BYTEScorrectly stayed in_writer.py-- it gates the strip / tile parallel-compress path, not overview generation. Easy thing to scoop accidentally; this version got it right._resolve_int_nodataplacement: the issue's scope note said "move only if overview-scoped." All three call sites are inside_block_reduce_2d, andgrepfinds no external callers. The move is correct.- Direct importers (
_writers/gpu.pyfor_MAX_OVERVIEW_LEVELSand_validate_overview_levels;_gpu_decode.pyfor_block_reduce_2d) were rewired to._overviewrather than relying on the re-export, which keeps the dependency edge explicit. The_gpu_decode.pydocstring cross-reference was updated to match. - The re-export in
_writer.pypreserves the public-ishxrspatial.geotiff._writerimport path that 17 test files rely on (test_cog.py,test_writer.py,test_polish_1488.py, the various*_overview_*and*_overview_nodata_*regression tests, etc.). No test-side churn needed. _writer.pydrops from 2238 to 1804 lines (-434), inside the 400-460 target.- The pre-existing
test_lowlevel_write_pushdown_2138.py::test_write_vs_to_geotiff_byte_parity_uint8[lz4]failure reproduces on a clean origin/main checkout, so it is not regression noise from this PR.
Checklist
- Behavior-neutral: byte-equivalent moved code
- Public import paths preserved via re-export
- All four backends (numpy / cupy / dask+numpy / dask+cupy) unaffected -- helpers are CPU-side, GPU dispatch unchanged
- NaN / nodata handling logic moved intact (issues #1613, #1623, #1975)
- Odd-shape ceil semantics moved intact (issue #2105)
-
_writer.pyline count drop within target (-434, target 400-460) - Sibling PR-J / PR-L scope respected -- only
_writer.py,_overview.py,_writers/gpu.py,_gpu_decode.pytouched
- Move the ``_overview`` re-export block up next to the existing ``_write_layout`` re-export, so both "lives elsewhere now" blocks sit together near the top of the module instead of one being mid-file. - Drop the ``OVERVIEW_METHODS`` ``#:`` docstring that was added during the move so the extraction is strictly content-identical to the original in ``_writer.py``.
brendancol
commented
May 21, 2026
Contributor
Author
brendancol
left a comment
There was a problem hiding this comment.
PR Review (follow-up pass)
Both nits from the first pass were addressed in b47f8579:
- The
from ._overview import (...)re-export now sits directly under the existing_write_layoutre-export block at_writer.py:102-118, so the two "lives elsewhere now" blocks read together. Matches the surrounding pattern. - The
OVERVIEW_METHODS#:docstring was dropped, so the moved content is now byte-equivalent to the origin/main version of_writer.py:352-812. I re-confirmed this withdiffagainst the source: the only delta is_PARALLEL_MIN_BYTES, which correctly stays in_writer.pybecause it gates the strip/tile parallel-compress path, not overviews.
Blockers
None.
Suggestions
None.
Nits
None.
What looks good
- Extraction is now strictly mechanical -- content-identical to origin/main, only the file boundary changed.
- Re-export shape preserves the 17 test-file imports from
xrspatial.geotiff._writerplus the_writers/gpu.pyand_gpu_decode.pydirect importers. _writer.pyfinal size 1800 lines (-438 from origin/main's 2238), inside the 400-460 target.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Mechanical extraction. Moves the overview pyramid helpers out of
_writer.pyinto a new_overview.pymodule so_writer.pyno longer carries pyramid logic alongside the strip / tile / IFD / layout code.Moved:
_make_overview_block_reduce_2d_replicate_pad_2d_resolve_int_nodata(overview-scoped; all call sites are inside the block reducer)_validate_overview_levelsOVERVIEW_METHODSand_MAX_OVERVIEW_LEVELS_writer.pyre-exports each name so the public import path (xrspatial.geotiff._writer) keeps working for tests, the_writerssubpackage, and_gpu_decode. Internal importers in_writers/gpu.pyand_gpu_decode.pyare switched to the new_overviewmodule directly._writer.pydrops from 2238 to 1804 lines (434 lines moved out, within the issue's 400-460 target).Test plan
pytest xrspatial/geotiff/tests/passes (5037 passed, 68 skipped; one pre-existing failure intest_lowlevel_write_pushdown_2138.py::test_write_vs_to_geotiff_byte_parity_uint8[lz4]reproduces on origin/main and is unrelated).from xrspatial.geotiff._writer import _make_overview, _block_reduce_2d, _replicate_pad_2d, _resolve_int_nodata, _validate_overview_levels, OVERVIEW_METHODS, _MAX_OVERVIEW_LEVELSstill works.from xrspatial.geotiff._overview import ...works for the same names.Closes #2259
Part of #2211