Experimental _pytest.scenario API: nested configs + in-memory collection (PoC)

[RonnyPfannschmidt]

Context

pytest's own testsuite is heavily integration-based: of ~3200 real test functions in testing/, ~61% use pytester, ~91% of those write files to disk, and roughly 1400 assertion sites glob-match rendered terminal output (vs ~270 structured ones). Tests need a full session because stdout is the only observable most of them have — there is no API to collect an item from an in-memory object, or to build a Config without a real rootdir/cwd. testing/conftest.py even reorders tests by decompiling __code__.co_names because "uses pytester" is the only available proxy for "slow".

This PR is a proof of concept for fixing that at the API level. Draft — the API shape and naming are up for discussion.

What's included

New experimental package _pytest/scenario/ (internal, not exported from pytest):

def test_static_method(tmp_path):
    class Test:
        @staticmethod
        def test_something(): ...
        @pytest.fixture
        def fix(self): return 1
        @staticmethod
        def test_fix(fix): assert fix == 1

    record = run_tests(Test, rootpath=tmp_path)
    record.assert_outcomes(passed=2)

• ConfigSpec / configured() — a parsed+configured nested Config from declarative data, built through the real parse phases but without rootdir discovery, config files, conftests, plugin autoload, or env consultation. Hermetic by default; teardown is paired via _ensure_unconfigure().
• fake_module(name, *members) — group loose functions/classes into an in-memory module with an explicit name; collect_tests() / run_tests() / stepwise scenario() collect through the standard pytest_collection flow (so -k/-m, parametrize, fixture scoping and pytest_collection_modifyitems all work) via a ScenarioModule whose _getobj serves the preset object. Nodeids derive from synthetic rootdir-relative paths that are never touched on disk.
• RunRecord / ItemRunRecord — typed results derived from real report objects via pytest_report_teststatus, with assert_outcomes() signature-compatible with pytester and per-test lookup (record["test_x"].setup.longreprtext).

Core changes in support:

• Config.parse() split into behavior-preserving phase methods (_preparse_addopts, _apply_rootdir, _register_core_ini_options, _load_plugins_phase, _load_initial_conftests_phase, _finalize_parse(decide_args=)), plus ArgsSource.SPEC. The programmatic path shares real code with command line parsing instead of bypassing it.
• Class.from_parent() no longer silently discards a passed obj (it was accepted and dropped).
• get_config() gained an explicit invocation dir= (was hardcoded Path.cwd()).
• debugging/unittest guard their cross-plugin option reads (trace/usepdb) so they work when the debugging plugin is not registered.

Validation: 33 self-tests in testing/test_scenario.py run in ~0.3s with hermeticity asserted (no sys.path/sys.modules/cwd/environ mutation, no files created); seven existing tests in testing/python/{collect,fixtures,metafunc}.py converted, including the hand-rolled make_function harness.

Deliberate scope cuts (documented follow-ups)

• The default scenario plugin set excludes capture, terminal, assertion, cacheprovider and the process-global-state plugins. capsys/capfd inside scenarios are unavailable until CaptureManager becomes stack-aware (suspend/resume is currently absolute, restoring values memoised at __init__ — nested managers work only by LIFO luck today). That is the next pillar.
• Conftest loading in scenarios (load_conftests=True) raises NotImplementedError; plugin objects via ConfigSpec.extra_plugins are the replacement.
• Host process warning filters (e.g. our own filterwarnings = error) are inherited by scenarios; a scenario's inicfg={"filterwarnings": [...]} takes precedence.
• Longer term: rebase pytester's inline_run onto this API, migrate stdout outcome-assertions to RunRecord, and consider pytest.scenario graduation.

The changelog fragment is 14809.feature.rst.

---

Per our AI contribution policy: this was researched, designed and implemented with Claude Code under my direction and review.

🤖 Generated with Claude Code