Definition fixture scope and a generic ItemDefinition node (#3926)

changelog/14769.bugfix.rst

@@ -0,0 +1,11 @@
+Selecting a parametrized test by its node id (``pytest test_x.py::test_a[1]``)
+now works when the test definition is a node of the collection tree
+(:confval:`collect_function_definition` set to ``pedantic`` or ``messy``). The
+argument matcher compares names level by level, and the definition node carries
+the bare name while the parametrization lives on the items below it; it now
+looks through the definition instead of failing to match.
+
+Relatedly, skipping a non-Python item whose ``reportinfo()`` reports no line
+number no longer crashes pytest with an ``INTERNALERROR``; the skip location is
+reported without a line instead. This became easy to hit now that non-Python
+definitions can carry ``pytest.param(..., marks=pytest.mark.skip(...))``.

changelog/3926.feature.rst

@@ -0,0 +1,32 @@
+Added the ``"definition"`` fixture scope, which shares one fixture instance
+across all invocations generated from a single test definition -- that is,
+across the parameter sets of one parametrized test, but not between two
+different tests:
+
+.. code-block:: python
+
+    @pytest.fixture(scope="definition")
+    def resource(): ...
+
+
+    @pytest.mark.parametrize("n", [1, 2, 3])
+    def test_it(resource, n):
+        # one `resource` for all three parameter sets
+        ...
+
+The scope needs the test definition to be a node of the collection tree, so it
+requires :confval:`collect_function_definition` to be set to ``pedantic`` (or
+``messy``). Using it without that fails with an explicit error rather than
+silently degrading to function scope.
+
+``@pytest.mark.parametrize(..., scope="definition")`` is supported as well and
+groups the invocations of one definition by parameter value.
+
+Added :class:`_pytest.nodes.ItemDefinition`, the collector-agnostic base for "a
+definition which generates items". Subclassing it lets a non-Python collector
+have its test definitions parametrized through :hook:`pytest_generate_tests`,
+exactly like Python test functions: declare the accepted names in
+``parametrize_argnames``, build one item per parameter set in ``make_item()``,
+and the chosen values arrive on ``item.callspec.params``. Parameter ids,
+``pytest.param(...)`` marks and node-id selection all work as usual; there is no
+fixture resolution on that path. See :ref:`non-python parametrization`.

changelog/3926.improvement.rst

@@ -0,0 +1,15 @@
+Added the :confval:`collect_function_definition` option, which can promote the
+internal ``FunctionDefinition`` to a real node in the collection tree.
+
+When set to ``pedantic``, a ``FunctionDefinition`` collector node is inserted
+between a :class:`~pytest.Module`/:class:`~pytest.Class` and its test functions,
+so that the (possibly parametrized) invocations of each test are collected
+underneath a shared definition node instead of as flat siblings. Function-level
+markers are scoped to the definition node in this mode.
+
+The default ``hidden`` keeps the previous flat layout, and the transitional
+``messy`` value keeps the definition node out of marker resolution for code that
+is not yet prepared for the new scope.
+
+The node ids of the individual test invocations are unchanged in all modes, so
+test selection, caching and reporting behave exactly as before.

changelog/3926.misc.rst

@@ -0,0 +1,7 @@
+Moved the parametrization core out of ``_pytest.python`` into the new
+``_pytest.parametrize`` module: ``CallSpec``, ``IdMaker`` and the new
+``ParametrizeContext``, of which :class:`pytest.Metafunc` is now the
+Python-specific subclass adding the fixture closure, signature validation and
+direct-parameter desugaring.
+
+``_pytest.python.CallSpec2`` keeps warning and resolving as before.

doc/en/example/nonpython.rst

@@ -105,3 +105,65 @@ interesting to look at the collection tree:
         <YamlItem ok>
 
     ======================== 2 tests collected in 0.12s ========================
+
+.. _`non-python parametrization`:
+
+Letting non-python tests take part in parametrization
+-----------------------------------------------------
+
+.. versionadded:: 9.2
+
+A collector which yields items directly, as ``YamlFile`` above does, produces a
+fixed set of tests. To let a test *definition* be parametrized -- by
+:hook:`pytest_generate_tests`, exactly like a Python test function -- collect
+:class:`~_pytest.nodes.ItemDefinition` nodes instead of items.
+
+Such a node declares the names it accepts in
+:attr:`~_pytest.nodes.ItemDefinition.parametrize_argnames` and builds one item
+per parameter set in :meth:`~_pytest.nodes.ItemDefinition.make_item`. The values
+chosen for a given item arrive on ``item.callspec.params``:
+
+.. code-block:: python
+
+    # content of conftest.py
+    from _pytest import nodes
+    import pytest
+
+
+    class YamlItem(pytest.Item):
+        def __init__(self, *, spec, **kwargs):
+            super().__init__(**kwargs)
+            self.spec = spec
+
+        def runtest(self): ...
+
+        def reportinfo(self):
+            return self.path, 0, self.name
+
+
+    class YamlDefinition(nodes.ItemDefinition):
+        parametrize_argnames = ("value",)
+
+        def make_item(self, parent, *, name, callspec, nodeid, context):
+            params = dict(callspec.params) if callspec is not None else {}
+            return YamlItem.from_parent(parent, name=name, nodeid=nodeid, spec=params)
+
+Any ``pytest_generate_tests`` implementation now applies:
+
+.. code-block:: python
+
+    def pytest_generate_tests(metafunc):
+        if "value" in metafunc.fixturenames:
+            metafunc.parametrize("value", ["good", "bad"])
+
+which collects ``test_simple.yaml::hello[good]`` and
+``test_simple.yaml::hello[bad]`` under a single ``<YamlDefinition hello>`` node.
+Parameter ids, ``pytest.param(...)`` marks and node-id selection all work as
+they do for Python tests.
+
+.. note::
+
+    This path covers parametrization only: there is no fixture resolution for
+    non-Python items, so ``indirect=True`` and requesting fixtures from such an
+    item are not supported. A ``parametrize()`` call naming anything outside
+    ``parametrize_argnames`` is rejected at collection time.

doc/en/how-to/fixtures.rst

@@ -389,7 +389,16 @@ to cause a ``smtp_connection`` fixture function, responsible to create a connect
 once per test *module* (the default is to invoke once per test *function*).
 Multiple test functions in a test module will thus
 each receive the same ``smtp_connection`` fixture instance, thus saving time.
-Possible values for ``scope`` are: ``function``, ``class``, ``module``, ``package`` or ``session``.
+Possible values for ``scope`` are: ``function``, ``definition``, ``class``,
+``module``, ``package`` or ``session``.
+
+.. note::
+
+    ``definition`` shares one fixture instance across all the invocations
+    generated from a single test definition -- that is, across the parameter
+    sets of one parametrized test, but not between two different tests. It
+    requires the definition to be a node of the collection tree, see
+    :confval:`collect_function_definition`.
 
 The next example puts the fixture function into a separate ``conftest.py`` file
 so that tests from multiple test modules in the directory can

doc/en/proposals/generic_fixture_closures.rst

@@ -0,0 +1,143 @@
+:orphan:
+
+============================================
+PROPOSAL: Generic fixture closures for items
+============================================
+
+.. warning::
+
+    This document is the follow-up half of the ``ItemDefinition`` work: it
+    outlines what is needed for *non-Python* items to request fixtures. The
+    parametrization half has shipped, see :ref:`non-python parametrization`.
+
+What shipped
+------------
+
+:class:`~_pytest.nodes.ItemDefinition` lets any collector take part in
+:hook:`pytest_generate_tests`. A definition declares the names it accepts up
+front:
+
+.. code-block:: python
+
+    class YamlDefinition(nodes.ItemDefinition):
+        parametrize_argnames = ("value",)
+
+        def make_item(self, parent, *, name, callspec, nodeid, context):
+            return YamlItem.from_parent(
+                parent, name=name, nodeid=nodeid, spec=dict(callspec.params)
+            )
+
+and the values chosen for each parameter set arrive on ``item.callspec.params``.
+
+``ParametrizeContext`` deliberately has no fixture machinery: ``_infer_scope()``
+returns ``Scope.Function``, ``_register_direct_params()`` is a no-op, and
+``_validate_argnames()`` checks against ``parametrize_argnames`` rather than a
+fixture closure. ``FixtureManager.pytest_generate_tests`` returns early for a
+context which is not a ``Metafunc``.
+
+That covers parametrization. It does **not** cover:
+
+- ``indirect=True`` -- there is no fixture to hand ``request.param`` to.
+- scope inference from fixture scopes -- an unspecified ``scope=`` is always
+  function scope.
+- a non-Python item requesting a fixture at all, including one at the new
+  ``"definition"`` scope.
+
+Why the rest is not a small step
+--------------------------------
+
+Fixture resolution is written against ``Function``, not against ``Item``. The
+concrete couplings:
+
+**TopRequest assumes a Python function item.**
+It is constructed as ``TopRequest(pyfuncitem=item)`` and reads
+``item._fixtureinfo``, ``item.funcargs``, ``item.fixturenames`` and
+``item.obj``. ``_fillfixtures()`` writes into ``item.funcargs``.
+``request.instance`` walks to a :class:`~pytest.Class` node and calls
+``newinstance()``.
+
+**getfixtureinfo() is item-scoped and signature-driven.**
+Its closure starts from the function's argument names
+(``getfuncargnames(func)``) plus autouse names. A non-Python definition has no
+signature; its "argnames" are whatever it declares. ``FunctionDefinition``
+already has to ``cast(nodes.Item, self)`` to call it -- the same wart, one level
+up.
+
+**Direct parametrization desugars into DirectParamFixtureDef.**
+That is how a direct param reaches the test at setup time. Without a fixture
+system there is nothing to desugar into, which is why the generic path delivers
+params on the callspec instead.
+
+**Function.setup() drives everything.**
+``nodes.Item.setup()`` is a no-op; the fixture lifecycle only runs because
+``Function.setup()`` calls ``self._request._fillfixtures()``.
+
+Proposed shape
+--------------
+
+Three steps, each independently useful.
+
+1. Split a node-level closure entry point out of ``getfixtureinfo()``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Today ``getfixtureinfo(node, func, cls, ignore_args)`` derives ``initialnames``
+from ``func``'s signature. Proposed:
+
+.. code-block:: python
+
+    def getfixtureinfo_for_argnames(
+        self, node: nodes.Node, argnames: Sequence[str], *, ignore_args=()
+    ) -> FuncFixtureInfo: ...
+
+with the current signature-based version becoming a thin wrapper which computes
+``argnames`` from the function. This removes the ``cast(nodes.Item, self)`` in
+``FunctionDefinition`` as a side effect, and gives ``ItemDefinition`` a closure
+without inventing a fake function.
+
+2. Make the request object item-generic
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Introduce a small protocol for what ``TopRequest`` needs from its item --
+``fixturenames``, somewhere to store the computed values, ``_fixtureinfo``, and
+an optional ``instance`` -- and give :class:`~pytest.Item` a default
+implementation. :class:`~pytest.Function` keeps its current behaviour
+(``funcargs``, bound instance); a generic item gets a plain dict and
+``instance is None``.
+
+The open question here is ``request.instance``, whose meaning is genuinely
+Python-specific. Returning ``None`` for non-Python items is the honest answer,
+but fixtures in the wild do use ``request.instance`` unconditionally.
+
+3. Opt in per definition
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Fixture support should not be forced on every ``ItemDefinition``; a collector
+which only wants parametrization should not pay for closure computation.
+Suggested switch:
+
+.. code-block:: python
+
+    class ItemDefinition(Collector, abc.ABC):
+        #: Whether items generated from this definition can request fixtures.
+        supports_fixtures: bool = False
+
+With ``supports_fixtures = True``, ``make_parametrize_context()`` computes a
+closure from ``parametrize_argnames`` via step 1 and returns a fixture-aware
+context, so ``indirect=True``, scope inference and ``"definition"`` scoped
+fixtures all work uniformly. ``FixtureManager.pytest_generate_tests`` would then
+key off the context being fixture-aware rather than off
+``isinstance(metafunc, Metafunc)``.
+
+Open questions
+--------------
+
+- Does ``Metafunc`` stay a distinct class once the generic context is
+  fixture-aware, or does it collapse into ``ParametrizeContext`` plus the
+  ``module``/``cls``/``function`` attributes?
+- ``request.instance`` for non-Python items: ``None``, or an error on access?
+- Autouse fixtures currently apply to every item in a subtree. Should they apply
+  to non-Python items which opted into fixtures, or only to those which declare
+  the corresponding names?
+- Should ``parametrize_argnames`` and the fixture closure be the same list, or
+  should a definition be able to accept a name for parametrization which is not
+  requestable as a fixture?