fix(web): rank summaries over whole groups and give every group one

[joseph-isaacs]

The bug

The random-access card ranked a single chart, not the group. collectRandomAccessSummary walked the group's chart links, returned the first that had rows, and published that chart's raw times under the group-wide title "Random Access Performance".

The producer decides how many charts that is: benchmarks/random-access-bench emits dataset as {dataset}/{pattern} plus a legacy bare taxi, so the group holds ~nine charts. The chart list is ORDER BY dataset, so the card was always feature-vectors/correlated — one chart of nine speaking for all nine. That is how lance was reported at 352us leading random access while it runs over 1ms on most of the other charts.

A second bug rode along: the SQL pinned every format to one global MAX(timestamp). lance runs less often than Vortex, so on any commit it skipped it dropped off the card entirely — the exact staleness the compression summaries already solve with their per-format snapshot policy.

Summaries were also gated off by default

Two hardcoded gates left benchmarks with no card at all:

• queryGroupHasV2Summary allowlisted five v2-era datasets (clickbench, statpopgen, polarsignals, tpch, tpcds). Everything the emitter has added since — spatialbench, fineweb, gharchive, appian, public-bi, clickbench-sorted, vortex — fell through to null.
• case 'VectorSearchGroup': return Promise.resolve(null) — no vector-search group ever had one.

Neither gate had a reason beyond "v2 didn't do it", and a newly added suite silently got nothing.

What changed

• Random access ranks every chart in the group, reading each format at its own newest run per chart so an intermittently benchmarked format stays on the card as of when it last ran instead of vanishing.
• One ranking model. Query, random-access, and vector-search summaries now share rankSeries: geomean of the damped (10 + value) / (10 + best) ratio per bucket, with v2's missing-series penalty where a series skipped a bucket. Query scores are unchanged — same code path, same 300us floor, and the pinned fixture scores (1.10554 / 1.60356) still hold.
• Coverage is reported. Each row carries measured/total, surfaced in the card's hover text, so a series measured in 4 of 9 charts says so rather than presenting a penalty-inflated score as like-for-like.
• Summaries are default-on. Both gates removed; collectGroupSummary's switch is exhaustive over GroupKey, so a sixth fact table becomes a compile error rather than a silently blank card. Recorded as a load-bearing rule in AGENTS.md.

Wire-shape change

The randomAccess summary variant's rows change from { name, time, ratio } to the shared { name, score, totalRuntime, measured, total }, and a new vectorSearch variant is added. This is the read API (GET /api/groups, GET /api/group/{slug}), not the producer→ingest contract, so SCHEMA_VERSION is untouched.

The card's value column now shows a geomean ratio and a total runtime instead of an absolute time, because no single absolute time is true of a nine-chart group. Worth a look during review — if absolute times are wanted, the honest alternative is a per-chart card, not a group one.

Checks

• pnpm test — 352 passed, 53 skipped
• tsc --noEmit, eslint ., next build — clean

The 53 skipped are the testcontainers Postgres suites, which include the four new random-access and vector-search cases added here; no Docker daemon was available in the authoring environment, so those ran only as mocked-pool equivalents in lib/summary.test.ts. CI should be checked for the real ones.

To cover that gap the new code path was additionally run end-to-end against a local Postgres 16 with the production schema (migrations/ applied), seeded with the nine-chart shape the producer actually emits — lance fast on feature-vectors/correlated, slower elsewhere, absent from two charts:

#1  vortex-file-compressed   1.13x   9.90 ms   (9 of 9 charts)
#2  lance                    2.23x  13.25 ms   (7 of 9 charts)
#3  parquet                  3.92x  34.20 ms   (9 of 9 charts)

Same 352us lance win on the first chart that produced the old card; it just no longer decides the ranking. Seeded data, so illustrative of the mechanism rather than a prediction of production values.

---

Generated by Claude Code

[github-actions]

Vercel preview for c0f3d9dfee5322169bb0c8cac44528e78c498c9e: https://benchmarks-website-77bddwk28-vortex-data.vercel.app