Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
### Unreleased

#### Bug fixes and improvements
- Promote the local COG read and write paths to the `stable` tier in `xrspatial.geotiff.SUPPORTED_FEATURES`. `SUPPORTED_FEATURES['writer.cog']` and `SUPPORTED_FEATURES['reader.local_cog']` now report `stable`; `reader.http_cog` stays `advanced` while the HTTP transport surface is contracted separately. The stable COG contract covers axis-aligned 2D / 3D rasters, the CPU writer and CPU reader, the lossless codecs (`none`, `deflate`, `lzw`, `zstd`, `packbits`), internal overviews, and normal CRS / transform / dtype / nodata / band / pixel-is-area / pixel-is-point round-trip. GPU COG paths, experimental codecs, rotated transforms, external `.tif.ovr` sidecars, file-like destinations with `cog=True`, BigTIFF COG, and HTTP COG remain outside the contract. Backed by the writer compliance suite (#2292), the cross-backend parity gate (#2293), and the per-tile byte-budget contract (#2294 / #2298). The reference docs (`docs/source/reference/geotiff.rst`) and the COG overview notebook spell out the full contract. (#2300)
- Resolve the GeoTIFF writer's `GeographicTypeGeoKey` / `ProjectedCSTypeGeoKey` decision via pyproj instead of an EPSG number range. The legacy heuristic (4326 + 4000-4999 -> geographic, else projected) silently mis-tagged geographic CRSes registered outside the 4000-4999 block (NAD83(2011) = 6318, GDA2020 = 7844, WGS 84 (G2139) = 9057, etc.) as projected and projected codes inside the block (4087 / 4088 / 4499) as geographic, corrupting the CRS at write time. The writer now calls `pyproj.CRS.from_epsg(...).is_geographic`. When pyproj can't classify a code (uninstalled, or installed but the local PROJ database lacks the entry), the writer raises the new `UnknownCRSModelTypeError` rather than guessing -- a small vetted allowlist (4326, 4269, 4267, 4258, 4283, 4322, 4230, 4019, 4047) is still honoured for the pyproj-missing case. `pyproj` is now listed under the `geotiff` extra. (#2277)
- Shut down the per-tile compression `ThreadPoolExecutor` on every exit path of the streaming tiled-write code in `to_geotiff`. The old code only called `shutdown(wait=True)` after the tile-row loop completed, so any mid-stream raise (compression failure, dask compute failure, file write failure) bypassed shutdown and leaked worker threads. The loop now runs inside `try/finally` and the finally calls `shutdown(wait=True, cancel_futures=True)` so queued tiles get dropped on the error path instead of blocking the unwind. The pool's workers carry an `xrspatial-geotiff-tile-compress` `thread_name_prefix` so leak-detection tests can tell them apart from dask's own offload/scheduler pools. (#2276)
- Remove read-side emission of the 13 deprecated GeoTIFF attrs (`crs_name`, `geog_citation`, `datum_code`, `angular_units`, `semi_major_axis`, `inv_flattening`, `linear_units`, `projection_code`, `vertical_crs`, `vertical_citation`, `vertical_units`, `colormap_rgba`, `cmap`) and bump `attrs['_xrspatial_geotiff_contract']` from 1 to 2. Downstream code that read these via `attrs[key]` now sees `KeyError`; migrate to `attrs.get(key)` or derive the value from `attrs['crs']` / `attrs['crs_wkt']` with pyproj. The `.xrs.plot()` accessor still surfaces palette colormaps by building a `ListedColormap` from the canonical `attrs['colormap']`. (#2016)
Expand Down
43 changes: 43 additions & 0 deletions docs/source/reference/geotiff.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,49 @@
GeoTIFF / COG
***************

Stable COG contract
===================

As of the #2286 production-readiness wave, the local COG read and write
paths are tagged ``stable`` in
:data:`xrspatial.geotiff.SUPPORTED_FEATURES`. ``SUPPORTED_FEATURES['writer.cog']``
and ``SUPPORTED_FEATURES['reader.local_cog']`` both report ``stable``;
``SUPPORTED_FEATURES['reader.http_cog']`` stays ``advanced`` while the
HTTP transport surface is contracted separately.

The contract covers:

* Axis-aligned 2D / 3D rasters.
* CPU writer and CPU reader paths.
* Stable codecs only: ``none``, ``deflate``, ``lzw``, ``zstd``,
``packbits``.
* Internal overviews only.
* Normal CRS, transform, dtype, nodata, band, and
pixel-is-area / pixel-is-point behavior.

The promotion is backed by the writer compliance suite (#2292), the
cross-backend parity gate (#2293), and the per-tile byte-budget contract
(#2294 / #2298). These tests run on every CI build so a regression in
the stable surface fails the build rather than silently shipping.

Outside the stable contract
----------------------------

The following combinations stay outside the stable contract. They still
work where they did before and are still tested, but they keep their
existing tier (``advanced``, ``experimental``, or ``internal_only``) and
the corresponding caveats:

* GPU COG read / write.
* Experimental codecs (``lerc``, ``jpeg2000`` / ``j2k``, ``lz4``).
* Internal-only ``jpeg``.
* Rotated transforms.
* External ``.tif.ovr`` sidecars.
* File-like destinations with ``cog=True``.
* BigTIFF COG (tracked separately).
* HTTP / range COG (tracked separately; see the byte-budget contract in
#2298).

Reading
=======
.. autosummary::
Expand Down
5 changes: 5 additions & 0 deletions examples/user_guide/52_COG_Overview_Generation.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@
"with `cog=True`, with no GDAL post-processing required."
]
},
{
"cell_type": "markdown",
"source": "### Stable COG contract\n\nThe local COG writer and the local COG reader are tagged `stable` in `xrspatial.geotiff.SUPPORTED_FEATURES` (`writer.cog` and `reader.local_cog`). Axis-aligned 2D / 3D rasters, the lossless codecs (`none`, `deflate`, `lzw`, `zstd`, `packbits`), internal overviews, and normal CRS / transform / nodata round-trip are covered by the parity and compliance suites that gate every CI build, so the examples in this notebook all sit inside the stable contract.\n\nA few combinations stay outside the stable contract and keep their existing `advanced` or `experimental` tier:\n\n- The HTTP COG reader (`reader.http_cog`) -- range fetching, redirect handling, and cache behaviour are not contracted yet. Use with care.\n- GPU COG read / write.\n- Experimental codecs (`lerc`, `jpeg2000` / `j2k`, `lz4`) and the internal-only `jpeg` codec.\n- Rotated transforms, external `.tif.ovr` sidecars, file-like destinations with `cog=True`.\n- BigTIFF COG (tracked separately).\n\nSee the *Stable COG contract* section of the GeoTIFF / COG reference page for the full list.",
"metadata": {}
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down
51 changes: 49 additions & 2 deletions xrspatial/geotiff/_attrs.py
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,34 @@
# them into a single ``cog`` key would tie a promotion of any one path to
# a promotion of all three. The keys carry no behaviour today; production
# code still gates on the underlying writer / reader options.
#
# Stable COG contract (issue #2300, closes the #2286 wave)
# --------------------------------------------------------
# As of #2300 the local COG paths (``writer.cog`` and ``reader.local_cog``)
# are tagged ``stable``. The promotion is backed by the writer compliance
# suite (#2292), the cross-backend parity gate (#2293), and the per-tile
# byte-budget contract that pins HTTP fetch behaviour (#2294 / #2298).
#
# The stable COG contract covers:
#
# * Axis-aligned 2D / 3D rasters.
# * CPU writer and CPU reader paths.
# * Lossless codecs: ``none``, ``deflate``, ``lzw``, ``zstd``, ``packbits``.
# * Internal overviews only (no ``.tif.ovr`` sidecars).
# * Normal CRS, transform, dtype, nodata, band, and
# pixel-is-area / pixel-is-point round-trip.
#
# The following combinations stay outside the stable contract and keep
# their existing tier:
#
# * GPU COG read / write (``writer.gpu``, ``reader.gpu``).
# * Experimental codecs (``lerc``, ``jpeg2000``, ``j2k``, ``lz4``) and the
# internal-only ``jpeg`` codec.
# * Rotated transforms (``reader.allow_rotated``).
# * External ``.tif.ovr`` sidecars (``reader.sidecar_ovr``).
# * File-like destinations with ``cog=True``.
# * BigTIFF COG (tracked separately).
# * HTTP / range COG (``reader.http_cog``); see the per-key comment below.
SUPPORTED_FEATURES = {
# Codecs. Tier 1 lossless integer + float byte-for-byte round-trip.
'codec.none': 'stable',
Expand Down Expand Up @@ -232,12 +260,31 @@
# single COG concept (which only carried ``writer.cog``) so the
# local and HTTP reader variants can promote independently of the
# writer and of each other.
'reader.local_cog': 'advanced',
#
# ``reader.local_cog`` is ``stable`` as of #2300: on-disk
# overview-IFD parsing is covered by the compliance suite (#2292)
# and the parity gate (#2293).
#
# ``reader.http_cog`` stays ``advanced``. The HTTP path layers
# range-request fetching, range coalescing, an SSRF / private-host
# filter, a per-tile byte-count cap, and partial-IFD handling on
# top of the local COG reader. The byte-budget contract (#2294 /
# #2298) pins per-tile fetch behaviour, but the transport-side
# surface -- redirect handling, network error reporting, cache /
# retry policy -- is not yet contracted at the ``stable`` bar.
# Tracked for separate promotion.
'reader.local_cog': 'stable',
'reader.http_cog': 'advanced',
'reader.gpu': 'experimental',
# Write paths.
'writer.local_file': 'stable',
'writer.cog': 'advanced',
# ``writer.cog`` is ``stable`` as of #2300: the CPU writer emits a
# spec-conforming COG layout (IFD-first, tiled, internal
# overviews, lossless codec) covered by the compliance suite
# (#2292) and the cross-backend parity gate (#2293). GPU writes,
# BigTIFF COG, file-like destinations, and experimental codecs
# stay outside the contract; see the block comment above.
'writer.cog': 'stable',
'writer.overviews': 'advanced',
'writer.bigtiff': 'advanced',
'writer.gpu': 'experimental',
Expand Down
Loading