From bf66dcf6eddf426b450608a8361463d00cd659c7 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Wed, 12 Aug 2026 15:59:41 -0400 Subject: [PATCH 1/3] Address feedback from third discussion round. --- peps/pep-0842.rst | 165 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 142 insertions(+), 23 deletions(-) diff --git a/peps/pep-0842.rst b/peps/pep-0842.rst index fd20b11c44b..e8991891c20 100644 --- a/peps/pep-0842.rst +++ b/peps/pep-0842.rst @@ -49,7 +49,8 @@ For example: This is **not** intended to be an access modifier for Python; see -:ref:`the rationale `. +:ref:`the rationale `. The mechanisms specified +by this PEP are easy to work around if necessary. Motivation @@ -207,7 +208,8 @@ importing modules that were also imported by that module. So, not only are users not prevented from accessing seemingly-public imports, they may be *encouraged* to do so by their language server! (This problem applies to any name that is meant to be private; it's just that imports are a particularly -common case for this to occur.) +common case for this to occur. For other examples, see :ref:`below +`) Real-world cases @@ -235,10 +237,26 @@ This caused a lot of breakage: * `Red Hat Bug 1583196 `__ +``requests.packages`` +^^^^^^^^^^^^^^^^^^^^^ + +The `requests `__ package +had an internal vendoring namespace that users treated as an API, so +after unvendoring packages, ``requests`` kept ``requests.packages`` +as an alias, which led to its own subtle breakage: + +* `psf/requests#3985 `__ +* `psf/requests#4102 `__ +* `psf/requests#4104 `__ +* `psf/requests#5327 `__ +* `psf/requests#5561 `__ +* `urllib3/urllib3#1518 `__ + + ``botocore.vendored`` ^^^^^^^^^^^^^^^^^^^^^ -The `botocore `__ package had vendored +The `botocore `__ package also had vendored dependencies under the ``botocore.vendored`` namespace, which ended up being `relied upon by users `__: @@ -273,6 +291,84 @@ and broke when they were removed in v0.23: * `Kaggle Product Feedback `__ +.. _pep-842-accidental-private-access: + +Other examples +************** + +Beyond imports, there are several examples where users accidentally accessed +internal APIs, which resulted in breakage. + + +``logging._acquireLock`` / ``logging._releaseLock`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The documentation for the :mod:`logging` module included a private API +in an example. This example was then copy-pasted to several downstream projects, +and was broken when the private APIs were removed in Python 3.13: + +- `sqlmapproject/sqlmap#5731 `__ +- `sqlmapproject/sqlmap#5796 `__ +- `conda/conda#14439 `__ +- `madphysicist/haggis#2 `__ +- `Debian Bug#1088763 `__ + + +``matplotlib.cbook._check_in_list`` / ``matplotlib.cbook._rename_parameter`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +`matplotlib `__ left some utility functions in a +module-level namespace. These fucntions were prefixed with a leading underscore, +but users disregarded this, leading to breakage when they were removed: + +- `matplotlib/matplotlib#18494 `__ +- `dougcahl/eddy_identification_winding#1 `__ +- `guchengxi1994/mask2json#58 `__ + + +``concurrent.futures.thread._threads_queues`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In Python 3.8, a recipe to make :class:`~conurrent.futures.ThreadPoolExecutor` +be killed by CTRL+C was spread around. This recipe used the internal API, and +was missed by many users (or potentially seen, but ignored, due to the issues +described :ref:`above `), leading to breakage in Python +3.9 when worker threads stopped being daemon: + +- `clchiou/non_graceful_shutdown.py `__ +- `python/cpython#83993 `__ +- `cognitedata/cognite-sdk-python#1122 `__ +- `Opentrons/opentron#12970 `__ + + +``re._pattern_type`` +^^^^^^^^^^^^^^^^^^^^ + +Before the existence of :class:`re.Pattern`, the type of objects returned by +:func:`re.compile` was private. Many users found it easier to access the internal +type rather than do ``type(re.compile(''))``, which led to breakage in 3.7 when +it was removed: + +- `beetbox/beets#2986 `__ +- `django-precise-bbcode#25 `__ +- `python/cpython#1646 `__ + + +``asyncio.staggered_race`` +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The `aiohappyeyeballs `__ package +(which is internally used by `aiohttp `__) used +the internal ``staggered_race`` API from the :mod:`asyncio` module. +This broke when the implementation was updated to no longer have a ``loop`` +parameter: + +- `aio-libs/aiohttp#8599 `__ +- `python/cpython#124639 `__ +- `python/cpython#124390 `__ +- `python/cpython#124700 `__ + + Linters cannot fight against imports ************************************ @@ -701,8 +797,8 @@ The grammar is changed to allow for the standalone ``export`` statement and | &"export" export_stmt -Note that augmented assignments (``x += y``) are disallowed through a PEG action -at compile time. +Note that augmented assignments (``x += y``), subscripts (``x[y] = z``), and +attributes (``x.y = z``) are disallowed through a PEG action at compile time. Standalone exports @@ -788,6 +884,8 @@ The following are NOT valid: export my: str, hovercraft: str = "full of", "eels" export name := "walrus" export hello += "world" + export trees[0] = "the larch" + export something.name = "python" Exporting functions and classes @@ -811,7 +909,7 @@ following is not valid: .. code-block:: python - def export name(): # NOT VALID + async def export name(): # NOT VALID ... class export Name: # NOT VALID @@ -1058,9 +1156,11 @@ Emitting a warning upon accessing unexported attributes This PEP initially proposed raising an :exc:`ImportError` upon accessing module attributes that were not listed in ``__export__``. This was not well received, as the PEP did not clearly describe the intentions behind -the proposal, and as such, many rejected the notion of "private attributes" -as a knee-jerk reaction. Following that feedback, the ``ImportError`` turned -into a warning, which was eventually determined to be a bad compromise. +the proposal. Following that feedback, the ``ImportError`` turned +into a warning, which was eventually determined to be a bad compromise, +as the ergonomics of warnings are much worse than exceptions, and because +many testing frameworks (such as ``pytest``) turn warnings into exceptions +during testing. Introduce ``__export__`` on its own @@ -1100,16 +1200,8 @@ For example: export def goodbye(self): print("Goodbye, world!") - -This was rejected primarily because it does not have a clear benefit over -the existing :ref:`name mangling behavior ` (using -the ``__`` prefix), which also solves many of the problems described in the -motivation of this PEP. - -Additionally, this is much more difficult to implement. The author's reference -implementation involved new access protocols, disabling optimizations, and overall -much more complexity when compared to the simple modification to the default -``module.__getattribute__`` behavior required by ``__export__``. +This is considered out of scope for this PEP, and may be revisited by a +future proposal. Add ``public`` and ``private`` decorators as builtins @@ -1192,20 +1284,39 @@ How should packages have access to their own private members? Imagine that a package has two modules: -1. ``library/_utils.py``, which is meant to contain utilities that are only +1. ``library/utils.py``, which is meant to contain utilities that are only for the developer of ``library``. 2. ``library/main.py``, which holds public APIs that are usable to the users of ``library``. -The names in ``_utils.py`` are not exported, because the module is not intended +The names in ``utils.py`` are not exported, because the module is not intended to be accessed by users of ``library``. But, ``main.py`` should have access to these names; the current proposal would result in ``main.py`` getting an -``ExportError`` upon importing private names from ``_utils.py``. +``ExportError`` upon importing private names from ``utils.py``. -How should this be resolved? Is this necessary at all -- as in, should ``_utils.py`` +How should this be resolved? Is this necessary at all -- as in, should ``utils.py`` mark its utilities as exported, and ask that users don't import anything from it? +Does ``export`` need a top-level marker? +---------------------------------------- + +Usage of ``export`` affects the runtime behavior of all other names defined +in a module, so it has been argued that this can make maintenance more difficult +in some cases. For example, if a developer is unsure whether a module already +uses ``export``, they would have to search the module for it in order to know +whether it is safe to declare a public API as ``export`` without affecting +the rest of the code. + +As a solution, it was proposed to require ``export`` syntax to have some sort +of marker at the top of the module (such as an ``__export__ = []`` declaration +or a ``__future__`` import). This has not been decided upon yet, because it is +unclear whether the problem described above will actually turn out to be an +issue in practice; it is expected that many libraries will be consistent about +their usage of ``export``/``__export__`` internally, and thus it should not be +very difficult for a developer to know what kind of module they are working in. + + Acknowledgements ================ @@ -1221,6 +1332,14 @@ Paul Moore, Steve Dower, and Barry Warsaw. Change History ============== +* 12-Aug-2026 + + - Clarified whether the ``export`` statement works with subscripts and + attribute assignments. + - Added an open issue on whether ``export`` syntax should be necessary + at the top of the file. + - Added more examples for real-world cases. + * 11-Aug-2026 - Required ``__export__`` to always be a :class:`list` object. From 70cc8eb6e9ddfb8ad9fac25b8512af69cb583aad Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Wed, 12 Aug 2026 17:16:44 -0400 Subject: [PATCH 2/3] Fix typos. --- peps/pep-0842.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/peps/pep-0842.rst b/peps/pep-0842.rst index e8991891c20..a558bf0abe2 100644 --- a/peps/pep-0842.rst +++ b/peps/pep-0842.rst @@ -318,7 +318,7 @@ and was broken when the private APIs were removed in Python 3.13: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `matplotlib `__ left some utility functions in a -module-level namespace. These fucntions were prefixed with a leading underscore, +module-level namespace. These functions were prefixed with a leading underscore, but users disregarded this, leading to breakage when they were removed: - `matplotlib/matplotlib#18494 `__ @@ -329,7 +329,7 @@ but users disregarded this, leading to breakage when they were removed: ``concurrent.futures.thread._threads_queues`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In Python 3.8, a recipe to make :class:`~conurrent.futures.ThreadPoolExecutor` +In Python 3.8, a recipe to make :class:`~concurrent.futures.ThreadPoolExecutor` be killed by CTRL+C was spread around. This recipe used the internal API, and was missed by many users (or potentially seen, but ignored, due to the issues described :ref:`above `), leading to breakage in Python @@ -338,7 +338,7 @@ described :ref:`above `), leading to breakage in Python - `clchiou/non_graceful_shutdown.py `__ - `python/cpython#83993 `__ - `cognitedata/cognite-sdk-python#1122 `__ -- `Opentrons/opentron#12970 `__ +- `Opentrons/opentrons#12970 `__ ``re._pattern_type`` From af8e843ff406be41776232421e49e917ca8ee4a4 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Wed, 12 Aug 2026 17:30:17 -0400 Subject: [PATCH 3/3] Add a section on bypassing. --- peps/pep-0842.rst | 42 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-) diff --git a/peps/pep-0842.rst b/peps/pep-0842.rst index a558bf0abe2..5549f4121f8 100644 --- a/peps/pep-0842.rst +++ b/peps/pep-0842.rst @@ -1015,8 +1015,8 @@ This is not an access modifier ------------------------------ This PEP does not aim to be a mechanism for preventing access to private -attributes in modules. The ``ExportError`` can be bypassed (such as by -accessing attributes through the module's ``__dict__``). +attributes in modules. The ``ExportError`` can be bypassed and avoided; +see :ref:`below `. This is by design. Python does not include access modifiers as a language feature for a reason. To `quote `__ Eric Smith: @@ -1104,6 +1104,43 @@ in Python, names are public (as in, importable) by default, *except* when for ``export`` in Python is "make everything else private except for this name". +.. _pep-842-bypassing-export: + +Bypassing ``__export__`` +------------------------ + +As mentioned previously, this proposal is not meant to be an ironclad +shield around private variables. + +For prototyping, the simplest way to get around ``__export__`` is to simply +delete it: + +.. code-block:: python + + import module + + del module.__export__ + # All private variables in 'module' are now available + +Or, for a more granular workaround, append specific private names to ``__export__``: + +.. code-block:: python + + import module + + module.__export__.append("name_you_want") + +However, this approach modifies the ``__export__`` list globally, meaning that +enforcement inside other packages will also be disabled. To avoid this, access +private variables through the module's ``__dict__``: + +.. code-block:: python + + import module + + name_you_want = module.__dict__["name_you_want"] + + Reference Implementation ======================== @@ -1339,6 +1376,7 @@ Change History - Added an open issue on whether ``export`` syntax should be necessary at the top of the file. - Added more examples for real-world cases. + - Added a section in "How To Teach This" about how to bypass ``__export__``. * 11-Aug-2026