Decouple datashader from xarray-spatial#3641
Conversation
…, xarray-contrib#3640) - Delete canvas_like() from xrspatial/utils.py (incl. the import datashader as ds try/except). canvas_like had no callers in xrspatial/ and was never exported. - Rewrite the suggest_zonal_canvas docstring example in xrspatial/zonal.py to use the arr.xrs.rasterize(...) accessor instead of ds.Canvas().polygons(). - Reword the pixel-center comment in xrspatial/terrain.py so it no longer names datashader. No public API change. Library source now has zero datashader imports.
…xarray-contrib#3459, xarray-contrib#3463, xarray-contrib#3464, xarray-contrib#3640) - fire.ipynb: shade(...) -> plt.imshow with LinearSegmentedColormap; pre/post NBR Images(...) -> side-by-side plt.subplots. - classification-methods example: drop datashader import; shade(...) with plt.get_cmap('terrain') -> plt.imshow(..., cmap='terrain'). - animated_hillshade.py: rewrite to use matplotlib colormaps + PIL alpha compositing instead of datashader shade/stack; build the terrain template as a plain xarray.DataArray (generate_terrain no longer takes a canvas= argument). Outputs cleared; source has zero datashader imports.
…ader (xarray-contrib#3453, xarray-contrib#3454, xarray-contrib#3455, xarray-contrib#3457, xarray-contrib#3640) - Add a shared _shade/_stack/_show/Elevation helper cell to each notebook; _shade maps an array through a matplotlib colormap to RGBA, _stack alpha-composites layers, _show displays (with optional background). - focal: shade/stack of terrain + hillshade composites -> helpers. - proximity: ds.Canvas().points/line -> .xrs.rasterize on a GeoDataFrame of Point/LineString geometries (merge='min' for ds.min); dynspread/ set_background -> _show(..., bg='black'). - classification: shade/stack -> helpers; Set1/Set3 -> matplotlib cmaps. - surface: Canvas grid setup -> generate_terrain on a template DataArray; the 2D-normal density grid -> np.histogram2d; the single observer point -> .xrs.rasterize on a Point GeoDataFrame; shade/stack/dynspread -> helpers. Outputs cleared; source has zero datashader imports.
…atashader (xarray-contrib#3456, xarray-contrib#3458, xarray-contrib#3460, xarray-contrib#3461, xarray-contrib#3462, xarray-contrib#3640) - zonal: ds.Canvas().line(..., agg=ds.sum('trail_segement_id')) -> .xrs.rasterize(line_gdf, column='trail_segement_id', merge='sum'); shade/stack -> helpers; rewrite 'Datashader aggregate' prose. - pathfinding: ds.Canvas().line/points -> .xrs.rasterize on Point / MultiLineString GeoDataFrames (split NaN-separated segments); dynspread/set_background -> _show(..., bg='black'). - multispectral: Canvas().raster(layer, agg='mean') resampling -> xarray .coarsen/.interp; shade/stack/Images/orient_array -> _shade/_stack/_show/_show_grid helpers. - Pathfinding_Austin example: ds.Canvas().line/points -> .xrs.rasterize on the geopandas GeoDataFrame (drop spatialpandas). - pharmacy-deserts: ds.Canvas().polygons/points with ds.mean/ds.max -> .xrs.rasterize(column=, merge=). merge='mean' is not a supported reducer, so non-overlapping census polygons use merge='last' (equivalent for a single covering polygon per pixel); counties use merge='max' to preserve the ds.max intent. inferno/Set1 -> matplotlib. Outputs cleared; source has zero datashader imports.
…array-contrib#3640) - setup.cfg: replace 'datashader >= 0.15.0' in the examples extra with matplotlib, geopandas, and shapely (the rendering and vector- rasterization deps the example notebooks now use). datashader is no longer pulled by 'pip install xarray-spatial[examples]'. - installation.rst: update the examples extra row to list matplotlib, geopandas, shapely; note datashader is no longer required. Deliberate datashader references kept: the A/B benchmark in benchmarks/rasterizer_benchmarks.py (import-guarded) and the historical- credit prose in why_xarray_spatial.rst / index.rst. Library source has zero datashader imports. Closes the consolidated decoupling effort for xarray-contrib#3451, xarray-contrib#3453-xarray-contrib#3464.
…ray-contrib#3640) Locks in that canvas_like() stays removed and that importing xrspatial.utils does not load datashader, guarding against accidental re-introduction of the library-source datashader dependency.
Melissari1997
left a comment
There was a problem hiding this comment.
PR Review: Decouple datashader from xarray-spatial
Reviewed the diff against main (18 files). The change removes datashader from library source, all 9 user-guide notebooks, 3 example notebooks/script, and the examples packaging extra, with no public API changes. The approach (substitute template.xrs.rasterize(...) for vector rasterization, matplotlib _shade/_stack/_show helpers for rendering, xarray .coarsen/.interp for raster resampling) is sound and matches the consolidated plan.
Blockers (must fix before merge)
None.
Suggestions (should fix, not blocking)
-
examples/animated_hillshade.py:115-124—_compositehas a dead alpha branch. Line 120 reassignsbaseto an RGB array (top[..., :3] * a + base[..., :3] * (1 - a), shape(H, W, 3)), so the guard on line 121 (base.shape[-1] == 4) is always false and the output alpha is hard-coded to 255. The visual output happens to look right because the terrain/water base layers are fully opaque, but the compositing is incorrect for any layer stack with a transparent base. The notebook_stackhelper (e.g.focal.ipynb,classification.ipynb) does this correctly by keepingrgbseparate and rebuildingbasewithnp.concatenate([rgb, out_a[..., None]], axis=-1). Recommend aligning_compositewith that implementation so the alpha math is right and the dead branch goes away. -
examples/pharmacy-deserts.ipynbcell that rasterizes block groups — the comment saysmerge='mean'is "equivalent to datashader'sds.mean('PCT_ABOVE_65')here" because block groups tile the plane. That equivalence holds only if polygons do not overlap and each pixel is covered by exactly one polygon. Census block groups are designed to tile, so this is correct in practice, but worth a one-line note that the equivalence relies on non-overlap (the current comment is close but could state the assumption more sharply). No code change needed if the comment is tightened.
Nits (optional improvements)
-
xrspatial/zonal.pydocstring example —ys = np.linspace(y_range[0], y_range[1], height)uses ascending y. The originalds.Canvasy-orientation differs, but theactual_min_pixelscount assertion is invariant to a y-flip so the doctest still passes if it were run. No action required; flagging only so a future reader doesn't assume the y-direction was carefully matched. -
The
_shadehelper is duplicated verbatim across 8 notebooks. This is acceptable for self-contained notebooks, but if a shared notebook helper module ever exists, these would be good candidates to centralize. Not blocking. -
examples/animated_hillshade.py:7docstring says "Requires: numpy, xarray, matplotlib, pillow." — also needsxrspatial(imported on line 16). Trivial docstring fix.
What looks good
canvas_like()removal is clean — it had no callers inxrspatial/and was never exported, and the regression test (test_canvas_like_removed_3451_3640) guards against re-introduction, including checking thatimport xrspatial.utilsdoes not load datashader viasys.modules.- The packaging change correctly adds
matplotlib,geopandas,shapelyto theexamplesextra (the deps the migrated notebooks actually use) rather than just dropping datashader. - Deliberate datashader references (the A/B benchmark in
benchmarks/rasterizer_benchmarks.pyand the historical-credit prose) were correctly preserved. - The
merge=mapping for the multi-reducer cases is correct:ds.min→merge='min',ds.sum→merge='sum',ds.max→merge='max'. Theds.meancase is correctly flagged as unsupported and handled withmerge='last'on non-overlapping polygons. - All 11 notebooks are valid JSON with zero datashader imports; all Python files compile.
Checklist
- N/A — no algorithm changes (packaging/docs/internal cleanup only)
- N/A — no backend code changed; numpy/cupy/dask paths untouched
- N/A — no neighborhood operations changed
- Regression test added for the one source removal (
canvas_like) - N/A — no dask chunk changes
- No premature materialization introduced (notebook helpers operate on numpy arrays)
- N/A — no new benchmarkable function
- N/A — no new public function (README matrix unchanged, correctly)
- Docstring rewritten for
suggest_zonal_canvas; comment reworded interrain.py
…y-contrib#3640) - animated_hillshade.py: _composite had a dead alpha branch (line 120 reassigned base to RGB, so the base.shape[-1]==4 guard on line 121 was always false and output alpha was hard-coded to 255). Rewrite to keep rgb separate and rebuild base with np.concatenate, matching the _stack helper used in the notebooks. Also add xrspatial to the requires line in the docstring. - pharmacy-deserts.ipynb: sharpen the merge='last' comment to state the non-overlap assumption (each pixel covered by exactly one polygon) that makes it equivalent to ds.mean.
Melissari1997
left a comment
There was a problem hiding this comment.
Follow-up review (after review-driven fixes)
Commit c2a85c85 addresses the two Suggestions and the actionable Nit from the first review.
Disposition of original findings
-
Suggestion 1 —
animated_hillshade.py_compositedead alpha branch: fixed. Rewrote_compositeto keeprgbseparate and rebuildbasewithnp.concatenate([rgb, out_a], axis=-1), matching the notebook_stackhelper. The output alpha is now tracked correctly through the layer stack instead of being hard-coded to 255. -
Suggestion 2 —
pharmacy-deserts.ipynbmerge='last'comment: fixed. Sharpened the comment to state the non-overlap assumption ("each pixel is covered by exactly one polygon") that makesmerge='last'equivalent tods.meanfor the census block groups. -
Nit —
animated_hillshade.pydocstringRequires:line: fixed. Addedxrspatialto the requires list. -
Nit — duplicated
_shadehelper across 8 notebooks: dismissed. Self-contained notebooks are the project convention (each user-guide notebook runs standalone without importing a shared helper module). Centralizing would require a new support module shipped with the docs, which is out of scope for this decoupling effort. -
Nit —
yslinspace direction in thezonal.pydocstring: dismissed. Theactual_min_pixelscount assertion is invariant to a y-flip, and the accessor handles orientation internally. No correctness impact.
What looks good now
- All review findings are either fixed or dismissed with a recorded reason.
animated_hillshade.pycompiles;pharmacy-deserts.ipynbis valid JSON.- No new findings from the follow-up diff.
Closes #3640. Implements the consolidated datashader decoupling across #3450, #3451, and #3453–#3464 (#3452 was already closed).
What changed
canvas_like()fromxrspatial/utils.py(the only library-source datashader import; no callers, never exported). Rewrote thesuggest_zonal_canvasdocstring example to usetemplate.xrs.rasterize(gdf, column='id', merge='max'). Reworded the pixel-center comment inxrspatial/terrain.py.Canvas().points/line/polygons+ds.mean/max/sum/min) →template.xrs.rasterize(gdf, column=, merge=)on GeoDataFrames. Rendering (shade/stack/dynspread/set_background+Elevation/Set1/Set3/inferno) → matplotlib_shade/_stack/_showhelpers.Canvas().raster(agg='mean')resampling → xarray.coarsen/.interp;orient_array/Images→ the helpers.merge='mean'is not a supported reducer, so non-overlapping census polygons usemerge='last'(equivalent for a single covering polygon per pixel); counties usemerge='max'to preserve theds.maxintent.datashader >= 0.15.0in theexamplesextra withmatplotlib,geopandas,shapely. Updatedinstallation.rstto match.Backend coverage
No backend code changed — this is a packaging/docs/internal-cleanup effort with no change to any public
xrspatial.*function. numpy / cupy / dask+numpy / dask+cupy paths are untouched.Test plan
test_canvas_like_removed_3451_3640regression test added — assertscanvas_likestays absent andimport xrspatial.utilsdoes not load datashader.pytest xrspatial/tests/test_utils.py test_terrain.py test_zonal.py -k "not dask"— 139 passed, 42 skipped (dask backends skipped because dask is not installed in this env; unrelated to the change).xrspatial/source has zero datashader imports (only an explanatory docstring comment inzonal.py).animated_hillshade.pycompiles.Deliberate datashader references kept
benchmarks/rasterizer_benchmarks.py(import-guarded viaHAS_DATASHADER).docs/source/getting_started/why_xarray_spatial.rstanddocs/source/index.rst.Notes
CHANGELOG.mdwas not touched (per rockout rules; maintained separately at release time).