Add a wrapper script that runs every test suite

[prestoncabe]

Closes #417.

What

Adds bin/run-all-tests, a single entry point for every automated test suite in the repository, wired up as devbox run test.

Suites run in this order:

1. builder-api — JVM tests (mvn test)
2. library-api — JVM tests (mvn test)
3. e2e — bin/run-e2e-tests, which boots the dev stack, runs the library-api Bruno collection, then the Playwright suite

The JVM suites go first because they need nothing but Maven, while the e2e suite takes exclusive ownership of the local service ports and swaps the root emulator-data directory for the e2e fixtures.

By default every selected suite runs even when an earlier one fails, so one invocation reports the complete picture. --fail-fast stops at the first failure. Individual suites can be selected by name. The script re-enters the Devbox environment on its own when invoked from outside it.

devbox run test
devbox run test -- --fail-fast
bin/run-all-tests builder-api library-api

Every run ends with a summary table of per-suite status and duration, and exits non-zero if any suite failed.

Setup now verifies through the wrapper

bin/setup previously verified the local environment with bin/run-e2e-tests, covering only the Playwright and Bruno suites. It now runs bin/run-all-tests --fail-fast, so the JVM suites are part of the verification gate too — and a failing JVM suite no longer costs you the several-minute e2e run.

Setup's mvn clean package steps gained -DskipTests. The wrapper owns the test run now, so without that change setup would run both JVM suites twice.

Testing

bin/run-all-tests was run end to end against a clean local stack; all three suites pass. bin/setup was also re-run end to end for the follow-up change.

🤖 Generated with Claude Code

[heyoub]

One other thing I ran into: my first library-api run failed because port 8081 was already occupied by the local service graph. The subsequent e2e runner stopped process-compose, after which the library tests passed. Since the wrapper intentionally runs the JVM suites before e2e, should it preflight/handle an already-running dev stack before starting those suites?

[heyoub]

I think the Devbox entrypoint is dropping arguments here. I ran devbox run test -- builder-api, but the wrapper reported Test suites selected: builder-api library-api e2e and ran all three suites. Running the wrapper directly with suite arguments appears to be the intended interface, so I think this needs to forward $@ through the Devbox script. This would also affect the documented devbox run test -- --fail-fast form.

[heyoub]

I hit a failure-path issue here as well. When Playwright fails, the HTML reporter starts its report server and waits at Press Ctrl+C to quit, so bin/run-e2e-tests never returns to this wrapper. That means run-all-tests never records the e2e failure or prints the promised final summary. I had to Ctrl+C manually. Should this invoke Playwright with the HTML reporter configured not to open/serve on failure?