[6.x] Stop tests depending on other tests having run first

[jasonvarga]

Cause

Our tests aren't isolated from each other. The suite runs as a single process, so every test file gets loaded, every static stays set, and every file a test writes into the testbench app is still sitting there for the next test. That hides two kinds of coupling.

Tests that leak. The testbench app at vendor/orchestra/testbench-core/laravel is created once and lives for the whole process. A sweep of all 984 test files found 198 leave files behind — ~50 into resources/, config/, addons/ and public/, 69 into storage/statamic/, 34 leaving Storage::fake residue, 29 leaving laravel.log. Two also wrote into the repo itself: MakeAddonTest rewrote the root package-lock.json, and DuplicateFormTest wrote tests/__fixtures__/users/.yaml.

Tests that depend on those leaks. Six files only pass because something else ran first:

• SelectTest uses helper classes declared at the bottom of PluckTest.php, so it needs that file loaded into the same process.
• ViewBlueprintListingTest asserts a custom namespace blueprint can be edited without ever creating one — StoreCustomBlueprintTest had left one in the skeleton.
• AssetTest's non-glideable upload test asserts storage/statamic/glide/tmp exists, but glide only creates it when it actually processes an image. It was inheriting the directory from an earlier test in the file.
• NavTest mocked Form::all() away entirely, with a TODO guessing that other tests were leaving behind forms without titles. It was simpler than that: any leftover form at all adds children to the Forms nav item and breaks the assertions.
• BardFieldtypeTest, MarkdownFieldtypeTest and PreparserTest need Antlers' guarded/allowed path lists on GlobalRuntimeState, which are only populated as a side effect of resolving the parser out of the container. resetGlobalState() doesn't clear them, so ParserTestCase — which builds its parsers by hand — ran against whatever the last test to resolve one left behind. Starting a process with them empty silently drops every modifier in user content, so {{ now format="Y" }} rendered a full datetime and a preparsed | upper did nothing.

This matters now because it blocks splitting the suite across processes, and it has already broken a sharded CI run. A leftover resources/forms/*.yaml containing {} has no title, so CoreNav calls Nav::item(null), gets null back, and tests/CP/Navigation/CoreNavTest.php dies with Call to a member function url() on null. In the single-process suite it's masked by lucky ordering.

Approach

Give every test a defined starting state rather than patching 198 tests to tidy up after themselves — the same idea as RefreshDatabase, applied to the other shared mutable state a test can reach.

Tests\RestoresTestbenchSkeleton, used by Tests\TestCase:

• Snapshots the skeleton once per process, just after the app boots.
• Deletes anything new after each test, deepest first.
• Prunes bootstrap/cache, storage/framework/{cache,views}, vendor and node_modules from the walk. The framework owns those — deleting them breaks every subsequent test with "Please provide a valid cache path" — and walking compiled views gets expensive.

A process starting against an already dirty skeleton would bake that dirt into its snapshot, so a new testbench.yaml declares the paths our tests are known to write and those are cleared before the first app boot. That's testbench's own purge config, so vendor/bin/testbench package:purge-skeleton cleans up after us too.

The individual tests are then made self-sufficient:

• MakeAddonTest — making an addon with a fieldtype runs npm install from the testbench app's base path. That app has no package.json, so npm walks up and installs against ours. Now fakes the process, as the other commands that trigger this already do.
• DuplicateFormTest — the users it makes have no id, so saving them writes .yaml. Now points the stache stores at the throwaway directory.
• ViewBlueprintListingTest — creates the blueprint it's asserting against.
• AssetTest — creates glide's temp directory up front and keeps the assertion that nothing lands in it.
• PluckTest — Item, ItemWithOrigin and ArrayAccessType move into their own files. Tests\ is autoloaded from tests/, so that's all they needed.
• NavTest — the Form::all() mock and its TODO are gone.
• ParserTestCase — resolves the real parser in setUp, so the Antlers runtime configuration comes from config every time instead of from whichever sibling happened to run first.

Worth calling out that ViewBlueprintListingTest and AssetTest started failing when the skeleton restore landed. They were green only on leaked state.

Results

	before	after
Full suite	9990 tests, 29900 assertions, 4:24	9990 tests, 29896 assertions, 3:56
Test files leaving the skeleton dirty	198 of 984	1
Test files writing into the repo	2	0
Test files failing when run on their own	5	0

The suite got faster rather than slower. The 4 missing assertions are the redundant assertDirectoryExists removed from AssetTest's 4 data sets. (AssetTest isn't in the isolation count — it passed as a whole file and only broke once tests stopped sharing state within a file.)

The one remaining leaver is tests/Tags/GlideTest.php, which configures a filesystem disk rooted at public/glide. The directory is created during the first test's app boot, before the snapshot, so an empty directory survives — and gets cleared at the start of the next process.

Also verified directly: planting a resources/forms/test.yaml containing {} in the skeleton no longer breaks CoreNavTest.

Two product bugs this turned up — Nav::create() returning null for a null display name, and a title-less form crashing the CP nav — are deliberately out of scope and will get their own PR.