diff --git a/peps/pep-0842.rst b/peps/pep-0842.rst index fd20b11c44b..5549f4121f8 100644 --- a/peps/pep-0842.rst +++ b/peps/pep-0842.rst @@ -49,7 +49,8 @@ For example: This is **not** intended to be an access modifier for Python; see -:ref:`the rationale `. +:ref:`the rationale `. The mechanisms specified +by this PEP are easy to work around if necessary. Motivation @@ -207,7 +208,8 @@ importing modules that were also imported by that module. So, not only are users not prevented from accessing seemingly-public imports, they may be *encouraged* to do so by their language server! (This problem applies to any name that is meant to be private; it's just that imports are a particularly -common case for this to occur.) +common case for this to occur. For other examples, see :ref:`below +`) Real-world cases @@ -235,10 +237,26 @@ This caused a lot of breakage: * `Red Hat Bug 1583196 `__ +``requests.packages`` +^^^^^^^^^^^^^^^^^^^^^ + +The `requests `__ package +had an internal vendoring namespace that users treated as an API, so +after unvendoring packages, ``requests`` kept ``requests.packages`` +as an alias, which led to its own subtle breakage: + +* `psf/requests#3985 `__ +* `psf/requests#4102 `__ +* `psf/requests#4104 `__ +* `psf/requests#5327 `__ +* `psf/requests#5561 `__ +* `urllib3/urllib3#1518 `__ + + ``botocore.vendored`` ^^^^^^^^^^^^^^^^^^^^^ -The `botocore `__ package had vendored +The `botocore `__ package also had vendored dependencies under the ``botocore.vendored`` namespace, which ended up being `relied upon by users `__: @@ -273,6 +291,84 @@ and broke when they were removed in v0.23: * `Kaggle Product Feedback `__ +.. _pep-842-accidental-private-access: + +Other examples +************** + +Beyond imports, there are several examples where users accidentally accessed +internal APIs, which resulted in breakage. + + +``logging._acquireLock`` / ``logging._releaseLock`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The documentation for the :mod:`logging` module included a private API +in an example. This example was then copy-pasted to several downstream projects, +and was broken when the private APIs were removed in Python 3.13: + +- `sqlmapproject/sqlmap#5731 `__ +- `sqlmapproject/sqlmap#5796 `__ +- `conda/conda#14439 `__ +- `madphysicist/haggis#2 `__ +- `Debian Bug#1088763 `__ + + +``matplotlib.cbook._check_in_list`` / ``matplotlib.cbook._rename_parameter`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +`matplotlib `__ left some utility functions in a +module-level namespace. These functions were prefixed with a leading underscore, +but users disregarded this, leading to breakage when they were removed: + +- `matplotlib/matplotlib#18494 `__ +- `dougcahl/eddy_identification_winding#1 `__ +- `guchengxi1994/mask2json#58 `__ + + +``concurrent.futures.thread._threads_queues`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In Python 3.8, a recipe to make :class:`~concurrent.futures.ThreadPoolExecutor` +be killed by CTRL+C was spread around. This recipe used the internal API, and +was missed by many users (or potentially seen, but ignored, due to the issues +described :ref:`above `), leading to breakage in Python +3.9 when worker threads stopped being daemon: + +- `clchiou/non_graceful_shutdown.py `__ +- `python/cpython#83993 `__ +- `cognitedata/cognite-sdk-python#1122 `__ +- `Opentrons/opentrons#12970 `__ + + +``re._pattern_type`` +^^^^^^^^^^^^^^^^^^^^ + +Before the existence of :class:`re.Pattern`, the type of objects returned by +:func:`re.compile` was private. Many users found it easier to access the internal +type rather than do ``type(re.compile(''))``, which led to breakage in 3.7 when +it was removed: + +- `beetbox/beets#2986 `__ +- `django-precise-bbcode#25 `__ +- `python/cpython#1646 `__ + + +``asyncio.staggered_race`` +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The `aiohappyeyeballs `__ package +(which is internally used by `aiohttp `__) used +the internal ``staggered_race`` API from the :mod:`asyncio` module. +This broke when the implementation was updated to no longer have a ``loop`` +parameter: + +- `aio-libs/aiohttp#8599 `__ +- `python/cpython#124639 `__ +- `python/cpython#124390 `__ +- `python/cpython#124700 `__ + + Linters cannot fight against imports ************************************ @@ -701,8 +797,8 @@ The grammar is changed to allow for the standalone ``export`` statement and | &"export" export_stmt -Note that augmented assignments (``x += y``) are disallowed through a PEG action -at compile time. +Note that augmented assignments (``x += y``), subscripts (``x[y] = z``), and +attributes (``x.y = z``) are disallowed through a PEG action at compile time. Standalone exports @@ -788,6 +884,8 @@ The following are NOT valid: export my: str, hovercraft: str = "full of", "eels" export name := "walrus" export hello += "world" + export trees[0] = "the larch" + export something.name = "python" Exporting functions and classes @@ -811,7 +909,7 @@ following is not valid: .. code-block:: python - def export name(): # NOT VALID + async def export name(): # NOT VALID ... class export Name: # NOT VALID @@ -917,8 +1015,8 @@ This is not an access modifier ------------------------------ This PEP does not aim to be a mechanism for preventing access to private -attributes in modules. The ``ExportError`` can be bypassed (such as by -accessing attributes through the module's ``__dict__``). +attributes in modules. The ``ExportError`` can be bypassed and avoided; +see :ref:`below `. This is by design. Python does not include access modifiers as a language feature for a reason. To `quote `__ Eric Smith: @@ -1006,6 +1104,43 @@ in Python, names are public (as in, importable) by default, *except* when for ``export`` in Python is "make everything else private except for this name". +.. _pep-842-bypassing-export: + +Bypassing ``__export__`` +------------------------ + +As mentioned previously, this proposal is not meant to be an ironclad +shield around private variables. + +For prototyping, the simplest way to get around ``__export__`` is to simply +delete it: + +.. code-block:: python + + import module + + del module.__export__ + # All private variables in 'module' are now available + +Or, for a more granular workaround, append specific private names to ``__export__``: + +.. code-block:: python + + import module + + module.__export__.append("name_you_want") + +However, this approach modifies the ``__export__`` list globally, meaning that +enforcement inside other packages will also be disabled. To avoid this, access +private variables through the module's ``__dict__``: + +.. code-block:: python + + import module + + name_you_want = module.__dict__["name_you_want"] + + Reference Implementation ======================== @@ -1058,9 +1193,11 @@ Emitting a warning upon accessing unexported attributes This PEP initially proposed raising an :exc:`ImportError` upon accessing module attributes that were not listed in ``__export__``. This was not well received, as the PEP did not clearly describe the intentions behind -the proposal, and as such, many rejected the notion of "private attributes" -as a knee-jerk reaction. Following that feedback, the ``ImportError`` turned -into a warning, which was eventually determined to be a bad compromise. +the proposal. Following that feedback, the ``ImportError`` turned +into a warning, which was eventually determined to be a bad compromise, +as the ergonomics of warnings are much worse than exceptions, and because +many testing frameworks (such as ``pytest``) turn warnings into exceptions +during testing. Introduce ``__export__`` on its own @@ -1100,16 +1237,8 @@ For example: export def goodbye(self): print("Goodbye, world!") - -This was rejected primarily because it does not have a clear benefit over -the existing :ref:`name mangling behavior ` (using -the ``__`` prefix), which also solves many of the problems described in the -motivation of this PEP. - -Additionally, this is much more difficult to implement. The author's reference -implementation involved new access protocols, disabling optimizations, and overall -much more complexity when compared to the simple modification to the default -``module.__getattribute__`` behavior required by ``__export__``. +This is considered out of scope for this PEP, and may be revisited by a +future proposal. Add ``public`` and ``private`` decorators as builtins @@ -1192,20 +1321,39 @@ How should packages have access to their own private members? Imagine that a package has two modules: -1. ``library/_utils.py``, which is meant to contain utilities that are only +1. ``library/utils.py``, which is meant to contain utilities that are only for the developer of ``library``. 2. ``library/main.py``, which holds public APIs that are usable to the users of ``library``. -The names in ``_utils.py`` are not exported, because the module is not intended +The names in ``utils.py`` are not exported, because the module is not intended to be accessed by users of ``library``. But, ``main.py`` should have access to these names; the current proposal would result in ``main.py`` getting an -``ExportError`` upon importing private names from ``_utils.py``. +``ExportError`` upon importing private names from ``utils.py``. -How should this be resolved? Is this necessary at all -- as in, should ``_utils.py`` +How should this be resolved? Is this necessary at all -- as in, should ``utils.py`` mark its utilities as exported, and ask that users don't import anything from it? +Does ``export`` need a top-level marker? +---------------------------------------- + +Usage of ``export`` affects the runtime behavior of all other names defined +in a module, so it has been argued that this can make maintenance more difficult +in some cases. For example, if a developer is unsure whether a module already +uses ``export``, they would have to search the module for it in order to know +whether it is safe to declare a public API as ``export`` without affecting +the rest of the code. + +As a solution, it was proposed to require ``export`` syntax to have some sort +of marker at the top of the module (such as an ``__export__ = []`` declaration +or a ``__future__`` import). This has not been decided upon yet, because it is +unclear whether the problem described above will actually turn out to be an +issue in practice; it is expected that many libraries will be consistent about +their usage of ``export``/``__export__`` internally, and thus it should not be +very difficult for a developer to know what kind of module they are working in. + + Acknowledgements ================ @@ -1221,6 +1369,15 @@ Paul Moore, Steve Dower, and Barry Warsaw. Change History ============== +* 12-Aug-2026 + + - Clarified whether the ``export`` statement works with subscripts and + attribute assignments. + - Added an open issue on whether ``export`` syntax should be necessary + at the top of the file. + - Added more examples for real-world cases. + - Added a section in "How To Teach This" about how to bypass ``__export__``. + * 11-Aug-2026 - Required ``__export__`` to always be a :class:`list` object.