ci: shard the test run and deflake timing-sensitive suites

[elkaix]

What

Splits the single tests CI job into 4 parallel shards so the suite finishes in a fraction of the wall-clock time, and loosens several timing-sensitive assertions that were flaking under CI load.

Changes

• .github/workflows/ci.yml: add a 4-shard strategy matrix (--shard I/4), fail-fast: false, and lower the per-shard timeout to 15 min.
• tests/run.sh: add --list and --shard I/N modes with a cost-based greedy bin-packing assignment (static per-suite weights) so each shard carries a similar load.
• tests/lease.sh: accept any wait_elapsed=<Ns> value instead of requiring exactly 0s.
• tests/liveness.sh: raise several wait_bounded ceilings (7→25s, 10→30s) so slow CI runners don't spuriously fail timing bounds.

Verification

• bash tests/run.sh --list enumerates all 18 suites.
• bash tests/run.sh --shard 1/4 through --shard 4/4 partition the inventory with balanced estimated loads; union of shards covers every suite exactly once.
• Full local run via bash tests/run.sh still passes end-to-end.
• CI on this PR exercises the new sharded matrix on macos-15.

Notes

• Shard assignment is deterministic (static costs, sorted insertion) — no cross-shard state or test reordering.
• Existing single-command usage (bash tests/run.sh or with explicit suite args) is unchanged.

Summary by CodeRabbit

• Tests

  • Improved test-suite discovery and execution, including suite listing, explicit selection, and validated parallel sharding.
  • Updated timing expectations to better accommodate slower environments and variable wait durations.
  • Preserved timeout reporting while improving distribution of test workloads.

• Chores

  • CI now runs tests across four parallel macOS shards with clearer shard status and shorter per-job time limits.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 84c03447-693f-40ff-b81f-c53770254211

📥 Commits

Reviewing files that changed from the base of the PR and between 62cff90 and f7397d2.

📒 Files selected for processing (4)

• .github/workflows/ci.yml
• tests/lease.sh
• tests/liveness.sh
• tests/run.sh

---

📝 Walkthrough

Walkthrough

The test runner now discovers suites dynamically, supports explicit selection and validated sharding, and balances suites by configured cost. CI runs four macOS shards. Lease and liveness tests use more tolerant timing assertions.

Changes

Test sharding and timing

Layer / File(s)	Summary
Dynamic suite selection tests/run.sh	The runner loads the suite inventory, defines suite costs, prints usage, and validates --list, explicit suites, and --shard I/N arguments.
Cost-aware shard execution tests/run.sh, .github/workflows/ci.yml	Suites are assigned to the least-loaded shard by descending cost. CI runs four named macOS shards without fail-fast and passes each shard selector to the runner.
Timing-tolerant test assertions tests/lease.sh, tests/liveness.sh	The lease diagnostic assertion accepts numeric elapsed durations. Liveness tests use longer process wait limits.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant GitHubActions
  participant tests_run_sh
  participant TestSuites
  GitHubActions->>tests_run_sh: start macOS shard I/4
  tests_run_sh->>TestSuites: discover and cost-rank suites
  tests_run_sh->>TestSuites: execute suites assigned to shard I
  TestSuites-->>GitHubActions: report shard result

Loading

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch chore/ci-shard-and-deflake

---

_{Comment @coderabbitai help to get the list of available commands.}