Skip to content

Build: Declare typing-extensions as a core dependency#3616

Open
anxkhn wants to merge 1 commit into
apache:mainfrom
anxkhn:patch-15
Open

Build: Declare typing-extensions as a core dependency#3616
anxkhn wants to merge 1 commit into
apache:mainfrom
anxkhn:patch-15

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 7, 2026

Copy link
Copy Markdown
Contributor
# Rationale for this change

`typing_extensions` is imported unconditionally at import time, not under a
`TYPE_CHECKING` guard:

- `from typing_extensions import Self` in `pyiceberg/typedef.py`, a module imported
  transitively by nearly the whole package.
- `from typing_extensions import override` in ten catalog and IO modules
  (`pyiceberg/catalog/__init__.py`, `catalog/sql.py`, `catalog/rest/__init__.py`,
  `catalog/glue.py`, `catalog/dynamodb.py`, `catalog/hive.py`, `catalog/noop.py`,
  `catalog/bigquery_metastore.py`, `io/pyarrow.py`, `io/fsspec.py`).

`typing.Self` and `typing.override` are only in the standard library on Python 3.11+
and 3.12+ respectively, while PyIceberg supports Python 3.10 (`requires-python =
">=3.10.0,<4.0.0"`), so the third-party `typing-extensions` back-port is genuinely
required at runtime.

However, `typing-extensions` is declared only in the `dev` dependency group, not in
`[project.dependencies]`, so the distributed package under-declares a required import.
It happens to work today only because `pydantic` (a real runtime dependency) pulls
`typing-extensions` in transitively. If that transitive edge ever changes, a clean
install would fail at `import pyiceberg` with `ModuleNotFoundError: No module named
'typing_extensions'`.

`deptry` (already declared as a dev dependency in `pyproject.toml`) flags exactly this
as `DEP004` ("imported but declared as a dev dependency") at every one of those import
sites.

This moves `typing-extensions` into `[project.dependencies]` with a floor of `4.4.0`,
the release that added `override` (`Self` landed in `4.0.0`, so `4.4.0` is the lowest
version that provides both), and regenerates `uv.lock` accordingly.

# Are these changes tested?

Yes. A regression test `test_typing_extensions_is_a_core_dependency` is added to
`tests/test_version.py`. It reads the installed package metadata via
`importlib.metadata.requires("pyiceberg")` and asserts that `typing-extensions` is
present as a core dependency (a requirement with no environment marker, as opposed to
the marker-gated optional extras). It fails before this change and passes after.

`deptry .` reports zero `typing_extensions` findings after the change (11 before).
`make lint` and `uv lock --check` pass.

# Are there any user-facing changes?

No API or behavior changes. The only user-facing effect is a correctly declared
dependency: installing `pyiceberg` now pulls in `typing-extensions` directly instead
of relying on it arriving transitively through `pydantic`.

Notes for the ship step (not part of the PR)

  • No tracking issue exists, so no Closes # line. (Historical PR Fix dependency with deptry  #534 proposed the
    same move but was closed by the stale bot for inactivity, not rejected on merit;
    worth mentioning in a PR comment if a maintainer asks, but not required in the body.)
  • The changelog label cannot be set by an external contributor (no triage perm);
    this is a build/packaging fix with no user-visible API change, so a changelog entry
    is likely unnecessary. Leave it to maintainer discretion.
  • Before opening: rebase patch-15 onto the freshly fetched origin/main (this repo
    moves ~10 commits/1.5 days). If upstream bumped deps, re-run uv lock conflict
    resolution and make lint (uv-lock check) after the rebase.
  • Open with: gh pr create -R apache/iceberg-python --base main --head anxkhn:patch-15.

typing_extensions is imported unconditionally at module import time:
`Self` in pyiceberg/typedef.py (a module imported transitively by nearly
the whole package) and `override` in ten catalog and IO modules. Both are
top-level runtime imports, not guarded by TYPE_CHECKING, and typing.Self /
typing.override are only in the standard library on Python 3.11+ / 3.12+
while pyiceberg supports Python 3.10.

However typing-extensions was declared only in the dev dependency group, so
the distributed package under-declared a required import. It works today
solely because pydantic pulls typing-extensions in transitively; if that
transitive edge ever changes, `import pyiceberg` would fail with
ModuleNotFoundError.

Move it to [project.dependencies] with a floor of 4.4.0 (the release that
added `override`; `Self` landed in 4.0.0). deptry (already a dev dependency)
flags this as DEP004 at every import site. Regenerate uv.lock accordingly.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant