feat(evalboard): replace the turn-budget signal with time per passed task

[bai-uipath]

The consumer half of Proposal: optimizing for wall clock time instead of expected turns. The producer is coder_eval_uipath#86, which derives each task's expected wall clock from run history and stamps expected_seconds into run.json.

Safe to merge in either order: runs without the stamp render as unscored rather than erroring.

Screenshots

---

Why the turn budget had to go

Problem	Evidence
The numbers were unmaintained	147 commits in skills touched expected_turns; six revised a value
The seed was a best-case extreme	Bulk-seeded from min_turns_success, the luckiest run ever seen; 44.7% of passing budgeted tasks exceed budget on a healthy night
Post-seed values are guesses	49% are multiples of 5; 204 tasks share a copy-pasted 300 against a p90 actual of 52, so they are permanently green
Coverage holes sit on the slow tasks	249 of 1002 tasks have no budget and hold 34.7% of task-seconds; of the 25 slowest, 14 have no budget
A turn is not a unit of time	Seconds per visible turn: p10 5.2s, p50 8.1s, p90 15.5s, max 156s. A Read and a 20-minute deploy both count 1
Not comparable across harnesses	On identical tasks Codex uses 0.50x Claude's turns and scores 98.3% within budget against Claude Code's 77.3%
Nothing consumed it	One evalboard chart, no Slack line, no gate

What changes on the dashboard

• Home page: the "Within Expected Turns (%)" card becomes Time per Passed Task, a per-harness trend of seconds per passed task. Hovering a point gives that run's seconds and its within-expected share. There is no headline figure over the chart: those numbers belong to one run and the chart spans many.
• Task grid: a new sortable vs Expected column carries the ratio and the tint. Duration stays untinted, so a long task is never mistaken for a slow one. Sorting by ratio is a genuinely different ranking than sorting by duration: on the 2026-08-18 codex nightly the two top-tens share zero tasks, and the ratio sort surfaces a uipath-troubleshoot cluster the duration sort cannot see.
• Task detail and trends: the ratio is printed beside the duration (7m29s 3.1×) and gets its own column in the per-task run history, so the number is readable without hovering.
• Watchlist: the third attention segment becomes time-overage; "turn-overage offenders" becomes slow-task offenders, reading 2m18s / 1m22s expected instead of turn counts.

Ratios render to one decimal. The second decimal is fake precision: the baseline is a min over a handful of runs (p10 over ten), and a task's own night-to-night spread is wider than the digit it would add.

The dashboard derives no baseline of its own. It reads the expected_seconds the runner stamped, so a number rendered here still matches the Slack ping that announced that run, and an unscored task (no passing run yet on that harness, or a run predating the stamp) reads as unscored everywhere rather than as "on target".

Two bugs that only running it against real data exposed

Both were invisible to the tests and are fixed here with regression coverage.

The board disagreed with the runner about its own headline. A codex nightly that stamps 3m12s rendered as 1m17s. timePerPassedTaskForTasks counted mature-skipped rows in the denominator while the runner excludes them from both sides, so it divided real seconds by 681 carried-forward passes that never ran. withinExpectedTimeRateForTasks had the same leak and survived only by accident, because those rows carry no stamped line.

The delta compared across harnesses. Consecutive nightlies alternate harness, so a codex run was being measured against the previous claude-code run and reported ▼ 75% vs prev, which described the schedule rather than a change in speed. That delta is now gone from the card entirely, for the reason above.

One semantic change worth reviewing

withinExpectedTimeRateForTasks scores passing tasks only. The turn rate it replaces counted a budgeted failure as over budget. Two reasons that does not carry over: a task that crashed in 10 seconds did not blow a time budget, and folding failures in would make a pass-to-timeout regression read as an efficiency gain once the slow pass stopped counting. Failure is already the pass rate's job, and the seconds burned failing are already in the headline's numerator (total seconds of every task that ran, over the number that passed).

expected_turns is deprecated, not deleted

The field stays on RunLimits, accepted and ignored, marked deprecated, with a CE031 exemption that states why. Deleting it now would fail every one of the ~930 skills task YAMLs that still declare it, since RunLimits sets extra="forbid", and would break any external suite that sets it (this package publishes to public PyPI and has an ADOPTERS.md). Removal belongs in its own release once those YAMLs are cleaned, and keeping the field here means no release-and-pin chain is on this PR's critical path.

Every consumer is gone: the orchestrator's one-shot warning and its two call sites, reports_stats.expected_turns_overage, the HTML header badge, the markdown Run-time Note, and the per-row expected_turns / expected_turns_overage report fields. Docs updated to describe the derived line instead.

Testing done

• pnpm vitest run: 523 passed, 43 files. tsc --noEmit: clean. next build: clean.
• Run end-to-end against real stamped data, not fixtures. Twelve real nightlies were replayed locally, each stamped using only the history that preceded it, and the board driven off them. That is what surfaced both bugs above, and it is the only check that compares what the board renders against what the runner wrote.
• New regression tests for both bugs: mature-skipped passes leaving both sides of the ratio, and a mature-skipped pass not counting as within expected.
• Component tests for the new column: the ratio rendering, the tint buckets, Duration staying untinted, and that sorting by ratio produces a different order than sorting by duration.
• uv run pytest -m "not live and not lint": 4175 passed. Four failures are pre-existing on main in this environment and untouched by this branch: test_litellm_route (us. vs eu. Bedrock profile, no litellm file in the diff) and three test_reports_stats_nonfinite cases (cohens_d → statistics.stdev on NaN; verified against 3.12.5 as well, so it is environment-local rather than a version issue, and cohens_d is byte-identical to main).
• ruff format --check + ruff check: clean. Custom CE lint: 344 passed.

Left to do

• Not seen on a deployed instance. Everything above is local. The chart, the new column and the hover have not been rendered against the live blob-backed data source.
• Not exercised on mobile. The grid's card layout carries the ratio under the duration rather than as a fifth stat; that path is tested but never viewed on a real device.
• The branch is behind main by a few non-conflicting commits and needs an update before merge.
• Manual deploy after merge: workflow_dispatch on coder_eval_uipath/.github/workflows/deploy-evalboard.yml with ref: main. That workflow does not fire on merge.
• Optional follow-up, not in this PR: the harness legend renders twice on the home page, once under each chart, with identical contents. De-duplicating it means coupling the two chart components, so it is left alone.

🤖 Generated with Claude Code

[github-actions]

Claude finished @bai-uipath's task in 1m 17s —— View job

---

Review in progress

Todo List

• Read .github/code_review.md for review standards
• Read CLAUDE.md for project conventions
• Analyze the full PR diff
• Review changed files with full context
• Perform cross-file consistency checks
• Analyze what's missing
• Design-level scrutiny
• Post comprehensive review

Starting review of PR #125: feat(evalboard): replace the turn-budget signal with time per passed task...

[uipreliga]

Maybe let's not replace the current turn-budget signal, but add this PR (time budget) to it.

[uipreliga]

For the expected, maybe let's do bottom quartile (p25)? what is it now?

[uipreliga]

Review: coder_eval — pr:125 (56 files) axis:1,2,3,4,5,6,7,8

Scope: pr:125 (56 files) axis:1,2,3,4,5,6,7,8 · branch bai/wall-clock-evalboard · 40b1c92 · 2026-08-21T17:40Z · workflow variant

Change class: complex — 56 files, +1281/-1592, spanning three coupled surfaces: it deprecates a public RunLimits field and rips out every consumer across the orchestrator and four report modules, rewrites the evalboard's core efficiency metric (new lib/timing.ts, four page surfaces, one chart replaced), and takes a new dependency on an expected_seconds field stamped by a separate repo (coder_eval_uipath#86), i.e. a cross-repo contract change with a two-sided rollout.

Architecture, security and test health are excellent (10, 10, 9.7) and the turn-to-wall-clock swap is a sound design that lands cleanly on most surfaces, but the real risks sit at the new metric's edges — a carried-forward row rendering a fabricated green 0.0× verdict on the most-visited page, a run.json field removed out from under the deployed dashboard with only a manual redeploy to fix it, and time aggregates that count timeouts under a "passing" label — so this is fix-then-merge, not rework.

Summary

Axis	Score	🔴	🟠	🟡	🔵	Top Issue
1. Code Quality & Style	7.4 / 10	0	1	3	1	The run.json timing block is parsed into RunOverview but unread: timePerPassedTask and visibleTurns are write-only, tolerance is reimplemented as a constant, and the surrounding comments promise fallbacks/roles the code does not implement
2. Type Safety	9.2 / 10	0	0	1	3	The per-run unstable_cache key was not versioned even though the cached PerRun shape changed (expectedTurns → expectedSeconds, +timePerPassedTask), so stale entries blank the new time signals on the overview/trends/watchlist for up to 5 minutes
3. Test Health	9.7 / 10	0	0	0	3	pivotByHarness is never exercised with timePerPassedTask — the only metric the new wall-clock chart plots
4. Security	10 / 10	0	0	0	0	—
5. Architecture & Design	10 / 10	0	0	0	0	—
6. Error Handling & Resilience	9 / 10	0	1	0	0	The mature-skipped (carried-forward) zero-duration sentinel is guarded in the run aggregates and on Trends but not in the task-grid render paths or the watchlist time metrics — producing a fabricated "0.0×" verdict and silently suppressing the slow-task signal
7. API Surface & Maintainability	8 / 10	0	1	2	0	The new metric's only input (expected_seconds) has no producer in this repo, yet the task guide, tutorial, run.json schema doc and OSS-visible UI all present it as automatic
8. Evaluation Harness Quality	8.5 / 10	0	1	1	0	Dropping expected_turns from the run.json task_results[] contract blanks three surfaces of the currently-deployed dashboard, with no release-note signal and a manual redeploy as the only fix

Overall Score: 9 / 10 · Weakest Axis: Code Quality & Style at 7.4 / 10
Totals: 🔴 0 · 🟠 4 · 🟡 7 · 🔵 7 across 8 axes.

Blockers

1. [Axis 1] The run.json timing block is parsed into RunOverview but unread: timePerPassedTask and visibleTurns are write-only, tolerance is reimplemented as a constant, and the surrounding comments promise fallbacks/roles the code does not implement (evalboard/lib/runs.ts:945) — Two RunOverview* fields are populated for every task of every run in readRunOverview and consumed by nothing. grep -rn "visibleTurns\|timePerPassedTask" --include="*.ts" --include="*.tsx" over evalboard/ returns only the definitions, the two write sites, and test fixtures that set them to null — no read.

(a) visibleTurns — leftover from the removal. runs.ts:945 visibleTurns: number | null;, written at runs.ts:1072 visibleTurns: visibleTurnsFromRaw(t), by the 12-line exported helper runs.ts:987 export function visibleTurnsFromRaw(t: {. Its own docstring names the metric this PR deleted as its justification — runs.ts:984-985: // provably identical to the persisted field — so it backfills the "within / // expected turns" metric for runs written before \visible_turns` existed.Turn cells now render fromdisplayedTurns()inlib/turns.ts, so the field is fully redundant. It still carries 6 dedicated tests (lib/tests/runs.test.ts:295-326`).

(b) timePerPassedTask — unused on arrival. runs.ts:978 timePerPassedTask?: number | null;, written at runs.ts:1098 timePerPassedTask: data.timing?.time_per_passed_task ?? null,, fed by the new 9-field runs.ts:468 interface RawTiming {. The field's comment at runs.ts:977 promises // run predates stamping (the front page then falls back to the task rows). — there is no fallback: overview.ts:674 unconditionally computes timePerPassedTask: timePerPassedTaskForTasks(scoped.tasks) from the rows and never consults overview.timePerPassedTask. The RawTiming header comment at runs.ts:465-467 (\tolerance` is recorded by the runner so the dashboard can never disagree with the run it is describing) is contradicted the same way — toleranceis never read;timing.ts:76hardcodesexport const TIME_BUDGET_TOLERANCE = 1;`.

Fix: delete visibleTurns + visibleTurnsFromRaw (and their tests) since the metric they served is gone; and either wire overview.timePerPassedTask/RawTiming.tolerance into overview.ts as the actual runner-stamped source (which is what the comments promise, and what would let the board agree with the Slack ping) or drop both and the RawTiming interface. Either way the three comments must stop asserting behavior that does not exist.
2. [Axis 6] The mature-skipped (carried-forward) zero-duration sentinel is guarded in the run aggregates and on Trends but not in the task-grid render paths or the watchlist time metrics — producing a fabricated "0.0×" verdict and silently suppressing the slow-task signal (evalboard/app/runs/[id]/task-grid.tsx:654) — The new vs Expected column computes its ratio with no matureSkipped guard, on both render paths:

// task-grid.tsx:654 (desktop table)
const timeRatioValue = timeRatio(
    t.durationSeconds,
    t.expectedSeconds,
);
const timeTint = tintForTimeRatio(timeRatioValue);

and identically at task-grid.tsx:794 (mobile cards). run-view.tsx:598 passes filtered — which includes mature rows (they render a <MaturePill /> at task-grid.tsx:691) — so these rows reach the column.

A carried-forward row is documented in this repo as carrying a zero, not a null: lib/runs.ts:949 — "but it wasn't executed — so its 0-cost/0-duration row must be excluded from per-task averages"; lib/trends.ts:165 — "its row carries 0 cost / 0 duration and no turns". The PR's own fixture encodes it: lib/__tests__/overview.test.ts:216-220 builds task({ expectedSeconds: 100, durationSeconds: 0, matureSkipped: true }).

So for a real mature row, timeRatio(0, 100) returns 0 — not null — and the cell renders fmtTimeRatioCell(0) = "0.0×" tinted text-emerald-700 (timeCellClasses("green")), with the tooltip expectedTimeTitle(100) = "expected time: 1m40s". A task that never executed is presented as having beaten its expected time by 100×, in green, on the most-visited page.

This PR establishes the correct treatment in the sibling surface — trends-view.tsx renders {e.matureSkipped ? "—" : fmtTimeRatioCell(timeRatio(e.durationSeconds, e.expectedSeconds))} with MATURE_TOOLTIP — and in the aggregates (lib/overview.ts:59 and :80), but not here.

Fix: gate both call sites the way trends-view.tsx does, e.g. const timeRatioValue = t.matureSkipped ? null : timeRatio(t.durationSeconds, t.expectedSeconds); and set the cell title to MATURE_TOOLTIP when t.matureSkipped. Add a task-grid.test.tsx case asserting a { matureSkipped: true, durationSeconds: 0, expectedSeconds: 100 } row renders — in vsExpCellFor(...) — the existing mature tests (task-grid.test.tsx:65, :107) never assert on that column, and the vs Expected suite (:124-157) never uses a mature row.
3. [Axis 7] The new metric's only input (expected_seconds) has no producer in this repo, yet the task guide, tutorial, run.json schema doc and OSS-visible UI all present it as automatic (docs/TASK_DEFINITION_GUIDE.md:314) — grep -rn 'expected_seconds' src/ tests/ returns nothing — coder-eval never writes expected_seconds or a run-level timing block into run.json; only the external eval runner does (evalboard/lib/runs.ts:405 points at eval_runner/skills/timing.py, a file in another repo). Yet the published guide states flatly at lines 316-321: "There is nothing to declare. A task's expected wall clock is derived... The eval runner computes it from past runs and stamps expected_seconds onto each row of run.json", and docs/tutorials/04-writing-a-task.md:65-67 tells a first-time task author "Nothing to declare for efficiency: expected wall clock is derived per task from run history and stamped into run.json after the run." Neither says that "the eval runner" is not part of coder-eval and that coder-eval run alone never produces the stamp. The UI compounds it: expectedTimeTitle (evalboard/lib/timing.ts:119) renders "no expected time yet (needs a passing run on this harness)" — a promise an OSS build can never fulfil no matter how many passing runs accumulate — and unlike the front-page chart (gated behind isInternal at evalboard/app/page.tsx:228) the new vs Expected column (task-grid.tsx COLUMNS, { key: "vsExp", header: "vs Expected" }) and the task-detail/trends ratios carry no edition gate, so they ship to the OSS edition as a permanently em-dashed column. Fix: state in both docs pages that expected_seconds is stamped by an external pipeline and that a standalone coder-eval run leaves the column unscored; and change the empty-state string to name the actual cause (e.g. "no expected time stamped for this task") instead of implying more passing runs will populate it.
4. [Axis 8] Dropping expected_turns from the run.json task_results[] contract blanks three surfaces of the currently-deployed dashboard, with no release-note signal and a manual redeploy as the only fix (src/coder_eval/reports_experiment.py:192) — eval_result_to_task_dict stops stamping two run.json task-row fields. The diff drops "expected_turns": expected_turns_value, (immediately after the surviving line 192 "visible_turns": visible_turn_count(result),) and "expected_turns_overage": list(overage) if overage is not None else None,. The PR verifies the forward direction (new dashboard vs old run.json = unscored, correct) but not the reverse, which is the one that actually happens on merge: the currently-deployed evalboard still reads expectedTurns: t.expected_turns ?? null and gates on const hasBudget = t.expectedTurns != null && t.expectedTurns >= 1; if (!hasBudget) continue; in the pre-PR turnBudgetRateForTasks. With the field gone from every new run.json, eligible is 0 for every run, the function returns null for all of them, and the front page's "Within Expected Turns (%)" chart plots nothing — plus every run-page "Turns" cell loses its tint. No error, no gap indicator, just a permanently empty headline chart. The author's own "Left to do" says the deploy workflow does not fire on merge and needs a manual workflow_dispatch; there is no evalboard deploy workflow in .github/workflows/ here to fire automatically. So that blank state starts at the first post-merge nightly and lasts until someone remembers to dispatch. Fix: either dispatch the evalboard deploy as a required step of merging this PR (say so in the PR body, not just in "Left to do"), or keep writing "expected_turns": expected_turns_value for one release so the two halves can be deployed in any order — which is what the "safe to merge in either order" claim would then actually mean for both directions rather than only for the producer PR.

Non-blocking, but please consider before merge

1. [Axis 1] The two new pass-scoped aggregates in overview.ts test the raw "SUCCESS" literal instead of the already-imported isPassStatus predicate (evalboard/lib/overview.ts:59) — Both new pass-scoped aggregates test the raw status string instead of the shared predicate:

• overview.ts:59 if (t.status !== "SUCCESS" || t.matureSkipped) continue;
• overview.ts:81 const passed = executed.filter((t) => t.status === "SUCCESS").length;

isPassStatus is already imported at overview.ts:20 (import { isPassStatus } from "./status";), and the same file carries an explicit rule against this at overview.ts:833-836:

            } else if (isPassStatus(t.status)) {
                // lib/status.ts, not a raw "SUCCESS" literal: `status` is an
                // untyped string, and this page's pass rate must move with
                // every other surface if the passing set ever widens.

The two new lines are exactly what that comment tells the next author not to write, and they sit in the numerator/denominator of the PR's two headline metrics — so if the passing set ever widens (lib/status.ts::statusCategory), the front-page time metrics would silently diverge from the pass rate beside them. Replace both with isPassStatus(t.status).
2. [Axis 1] The "2× expected" boundary has multiple independent encodings and the EVALBOARD_TIME_RED_RATIO knob moves the cell tint without moving the TIME_BUDGET_TOLERANCE rollup boundary, breaking the invariant the module's comment asserts (evalboard/lib/timing.ts:75) — timing.ts:75 asserts an invariant the code cannot hold: // run's \timing` block. Red tints at the same 2×, so the cell and the rollup agree.` But the two sides of that claim are separate values, and one is externally overridable:

• timing.ts:76 export const TIME_BUDGET_TOLERANCE = 1; → the rollup line is 1 + 1 = 2 (hard constant, withinExpectedTime, timing.ts:88).
• timing.ts:25 red: parse(process.env.EVALBOARD_TIME_RED_RATIO, 2), → the tint line is 2 by default only.

Set EVALBOARD_TIME_RED_RATIO=3 on a deployment and the cell tints red at 3× while withinExpectedTimeRateForTasks still counts anything past 2× as over — the stated "cell and the rollup agree" is gone, with nothing failing. Two more copies hardcode the same number as prose that the knob cannot move:

• app/_overview/wall-clock-chart.tsx:104 : \${r.toFixed(0)}% within 2× expected`;`
• app/runs/[id]/task-grid.tsx:48 vsExp: "… Past 2× counts as slow; …"

Fix: derive one value — e.g. export const RED_RATIO = 1 + TIME_BUDGET_TOLERANCE and have getTimeRatioThresholds() use it as the red default (or drop the red env override entirely), then interpolate that constant into both UI strings instead of typing 2×. Worth noting while you are there: EVALBOARD_TIME_YELLOW_RATIO / EVALBOARD_TIME_RED_RATIO are referenced nowhere outside timing.ts and lib/__tests__/timing.test.ts:65-85 — no deployment, doc, or .env sets them — so this is a knob that exists only to be a second source of truth.
3. [Axis 1] The turn-budget removal sweep left stale comments and retired vocabulary behind: an orphaned experiments/default.yaml comment now mis-annotates task_timeout, a TaskGrid comment describes removed openHelp state, and turn-budget naming survives in touched files including a new test (experiments/default.yaml:24) — The PR deleted the commented-out # expected_turns: 15 line but left the three-line comment that documented it, so it now reads as the preamble to task_timeout:

23:    max_turns: 20
24:    # Soft target: when cumulative SDK turns across all iterations of a task
25:    # exceed this, the orchestrator logs a one-shot warning and the report
26:    # surfaces a badge. Does NOT abort the run (max_turns is the hard cap).
27:    # Maximum total seconds for the entire task evaluation (all iterations combined).
28:    task_timeout: 600

This is the baseline defaults file every task inherits from (layer 1 of the 5-layer merge), and it now tells a reader that task_timeout "does NOT abort the run" — the opposite of what RunLimits.task_timeout does. Delete lines 24-26.
4. [Axis 2] The per-run unstable_cache key was not versioned even though the cached PerRun shape changed (expectedTurns → expectedSeconds, +timePerPassedTask), so stale entries blank the new time signals on the overview/trends/watchlist for up to 5 minutes (evalboard/lib/overview.ts:292) — This PR changes the shape of the object stored under this cache entry — RunOverviewTask.expectedTurns was removed and expectedSeconds added (runs.ts:944), and RunOverview gained timePerPassedTask (runs.ts:978) — but the key parts are unchanged:

    const loader = unstable_cache(
        (id: string) => loadPerRunForId(id, source),
        ["evalboard-per-run", source.id],
        { revalidate: 300 },
    );

Next's key is hashString([MAIN_KEY_PREFIX, fetchCacheKeyPrefix, ${cb.toString()}-${keyParts.join(',')}]) (node_modules/next/dist/server/web/spec-extension/unstable-cache.js:55,82-83 + incremental-cache/index.js:200-206). No build id participates, and cb.toString() here is the unchanged arrow (id: string) => loadPerRunForId(id, source) — so the key is byte-identical before and after this PR. That is precisely why the two sibling loaders in this codebase carry explicit version tokens and were bumped for smaller shape changes: ["aggregate-task-trends-v3", source.id] with the comment "the cached shape gained tagCounts … the key bump keeps a stale pre-deploy payload from being served into new code" (trends.ts:258-259) and ["history-for-task-v2", source.id] (trends.ts:339). A stale entry served into the new code yields a PerRun whose tasks[i].expectedSeconds is undefined while the type says number | null — it degrades to — rather than crashing, but the type is asserted over an unvalidated deserialized payload, and the within-expected tooltip and every vs Expected cell go blank for up to revalidate: 300. Bump to ["evalboard-per-run-v2", source.id].
5. [Axis 7] getTimeRatioThresholds() reads non-NEXT_PUBLIC_ env vars, so the new tint knobs are inert at the three client-component call sites (timing.ts:24-25) (evalboard/lib/timing.ts:24) — timing.ts:23-26 reads parse(process.env.EVALBOARD_TIME_YELLOW_RATIO, 1.5) / parse(process.env.EVALBOARD_TIME_RED_RATIO, 2), and tintForTimeRatio calls it as a default argument (line 51: t: TimeRatioThresholds = getTimeRatioThresholds()), evaluated at each call site. Two of the three call sites are client components: evalboard/app/runs/[id]/task-grid.tsx line 1 is "use client" and calls tintForTimeRatio(timeRatioValue) at lines 658 and 798; evalboard/app/trends/trends-view.tsx line 1 is "use client" and calls it at line 299. Next.js only inlines NEXT_PUBLIC_-prefixed vars into the client bundle (evalboard/next.config.mjs adds no env passthrough), so in the browser both reads are undefined and the knobs silently fall back to 1.5/2 — while the same helper on the un-use client task-stats.tsx honors them, giving two surfaces different thresholds from one setting, plus a hydration mismatch on the tint class if SSR resolves the var and the client does not. The codebase already documents this exact rule at evalboard/app/runs/[id]/run-view.tsx:174: "Edition gate (passed from the server page — process.env isn't readable in this client component)". Fix: resolve the thresholds once in the server page and pass them down as a prop (mirroring isInternal), or rename them NEXT_PUBLIC_EVALBOARD_TIME_*. Also document them in evalboard/README.md — neither name appears there or anywhere outside timing.ts and its test.
6. [Axis 7] Metric and chart help on the front page was demoted from visible text to native title tooltips, making the headline definition and the "scoped to the active filter" notice unreachable on touch and by keyboard (evalboard/app/page.tsx:286) — Three surfaces lose reachable help in one PR. (1) evalboard/app/page.tsx:285-291 deletes the visible <p className="text-xs text-gray-500"> caption that explained the chart and moves the whole definition into a hover-only attribute on a non-interactive heading: <h2 className="text-sm font-semibold text-gray-900" title="Total wall clock ÷ tasks passed. Every task's seconds count, failures included; only passes count in the denominator...">. The sibling "Daily Success Rate (%)" heading a few lines above keeps its visible caption, so the two charts are now inconsistent, and the counterintuitive half of this metric (failures in the numerator, passes in the denominator) is exactly the part now invisible. (2) evalboard/app/_components/col-help.tsx drops ColHelp / HelpPopover / ColHelpIcon and the causes / fix guidance entirely, leaving four plain strings. (3) evalboard/app/runs/[id]/task-grid.tsx replaces the per-column ⓘ button (which had aria-label={What is ${col.header}?} and aria-expanded) with title={COLUMN_HELP[col.key]} on the <th>. A title on a non-focusable element is not exposed to keyboard users, is not reliably announced by screen readers, and does not open on touch — and the PR states it was never viewed on a real mobile device. Fix: keep the one-line visible caption under the front-page heading (matching the neighbouring chart), and if the popovers are staying deleted, move the header help onto the existing sort <button> so it is at least focusable.
7. [Axis 8] Watchlist time aggregates include failed/timed-out tasks with no outcome filter, so the "Slow-task offenders" panel can list a skill whose every outcome timed out under the label "Passing, but well past their expected time" (evalboard/lib/watchlist.ts:194) — attention() collects the ratio with no outcome filter — const r = timeRatio(t.durationSeconds, t.expectedSeconds); if (r != null) ratios.push(r); at line 194 — and timeOverage() repeats it verbatim at line 340. Meanwhile the same PR establishes the opposite semantics for the headline in overview.ts:56: if (t.status !== "SUCCESS" || t.matureSkipped) continue;, justified in the comment as "a task that crashed in 10 seconds did not blow a time budget". Swapping turns for wall clock reverses the direction of that leakage: a crash used to deflate the turn ratio (2 turns against a budget of 8), but a task_timeout breach inflates the time ratio without bound (600s against a 60s line = 10×), so clamp01(mean(ratios) - 1) saturates at 1.0 and hands the row the full TIME_WEIGHT * 1.0 = 20 points that FAIL_WEIGHT * failRate already charged for the same failure. The user-visible strings are then untrue: line 165 returns "Passing, but well over expected time" and watchlist-view.tsx:520-521 titles the panel "🐌 Slow-task offenders" / "Passing, but well past their expected time" for rows that may be purely timeouts. Add the same isPass(t.status) guard both places (matching the headline), or change the three labels to stop claiming these are passing tasks.

Nits

1. [Axis 1] Two formatters render the same ratio two different ways on the same element — 2.63x expected (ASCII x) vs 2.6× (multiplication sign) (evalboard/lib/timing.ts:104) — timing.ts:105 return ratio == null ? "—" : \${ratio.toFixed(2)}x expected`;andtiming.ts:112 return ratio == null ? "—" : `${ratio.toFixed(1)}×`;differ in both precision and glyph.task-stats.tsxshows both on one
   : fmtTimeRatio(ratio)in the hover title at:32andfmtTimeRatioCell(ratio)in the visible chip at:41— so a task hovers as2.63x expectedabove a chip reading2.6×. timing.ts:109-110 justifies the one-decimal choice (One decimal, not two: the baseline is a min over a handful of runs), which argues the two-decimal variant should not exist at all. Collapse to one formatter (one decimal, ×) and let the caller append " expected"`.
2. [Axis 2] timing.ts carries two disagreeing definitions of "unscored": the ratio helpers reject expectedSeconds <= 0 while the title/stat helpers only check != null, so expected_seconds: 0 reads as a real 0-second target (evalboard/lib/timing.ts:117) — timeRatio (timing.ts:39-45) and therefore withinExpectedTime treat a non-positive stamp as unscored, but expectedTimeTitle uses a different predicate:

export function expectedTimeTitle(expectedSeconds: number | null): string {
    return expectedSeconds != null
        ? `expected time: ${fmtTaskSeconds(expectedSeconds)}`
        : "no expected time yet (needs a passing run on this harness)";
}

and ExpectedTimeStat repeats it at evalboard/app/runs/[id]/[...task]/task-stats.tsx:66 ({expectedSeconds != null ? fmtTaskSeconds(expectedSeconds) : "—"}). With a stamped expected_seconds: 0 (or a negative one), the vs Expected cell renders — while the very tooltip on that same cell (task-grid.tsx:706) asserts expected time: 0m00s, and the task page's "Expected time" stat prints 0m00s as if a line existed. Extract the guard into one exported predicate (e.g. hasExpectedTime(s): s is number => s != null && Number.isFinite(s) && s > 0) and use it in all three places, so the module's own header claim — "A task with no expected_seconds is unscored … Every helper returns null there" (timing.ts:9-10) — is actually true of every helper.
3. [Axis 2] Non-null assertions on durationSeconds / expectedSeconds rely on an invariant that lives inside another function and that TypeScript cannot verify (evalboard/lib/watchlist.ts:343) — timeOverage re-points these assertions at the new, externally-stamped field:

            const r = timeRatio(t.durationSeconds, t.expectedSeconds);
            if (r == null) continue;
            push(ratios, t.skill, r);
            push(seconds, t.skill, t.durationSeconds!);
            push(expected, t.skill, t.expectedSeconds!);

The assertions are sound only because timeRatio returns null when either input is null — an invariant in a different module that the compiler never re-checks, so a future relaxation of timeRatio (e.g. defaulting a missing duration) silently converts these into undefined values pushed into number[] arrays that mean() then turns into NaN. Narrow locally instead: const { durationSeconds: d, expectedSeconds: e } = t; if (d == null || e == null) continue; then compute the ratio from d/e — same control flow, zero !, and the compiler enforces it. Same applies to r.avgSeconds / r.avgExpectedSeconds consumers.
4. [Axis 2] The new vsExp sort comparator returns NaN when both rows are unscored, silently dropping the taskId tie-break (evalboard/app/runs/[id]/task-grid.tsx:303) — ```ts
case "vsExp":
return (
(timeRatio(a.durationSeconds, a.expectedSeconds) ?? -Infinity) -
(timeRatio(b.durationSeconds, b.expectedSeconds) ?? -Infinity)
);

When neither row is scored — which is *every* row on every run that predates the `expected_seconds` stamp, i.e. all history and every run until the producer PR lands — this evaluates `-Infinity - -Infinity` = `NaN`. The caller at task-grid.tsx:549-551 then does `const c = compare(a, b, sort.key); if (c !== 0) return sort.dir === "asc" ? c : -c; return a.taskId.localeCompare(b.taskId);` — `NaN !== 0` is `true`, so it returns `NaN` and the alphabetical tie-break is never reached; ECMA-262 SortCompare coerces NaN to +0, so clicking "vs Expected" leaves the grid in raw run.json order rather than the intended alphabetical order. The other columns share the `?? -Infinity` idiom (pre-existing), but they are populated in practice while this one is empty by default. Guard the sentinel case, e.g. compare via a helper that maps null to a sentinel and returns `0` when both are null, or use `Number.isNaN(c) ? a.taskId.localeCompare(b.taskId) : ...` in the caller.
5. **[Axis 3] `pivotByHarness` is never exercised with `timePerPassedTask` — the only metric the new wall-clock chart plots** (`evalboard/app/_overview/__tests__/harness-series.test.ts:23`) — The PR widens `HarnessMetric` to three members (`harness-series.ts:15-18`), but the test factory hardcodes the new one to null —
```ts
        successRate,
        withinExpectedTimeRate,
        timePerPassedTask: null,

— and both metric-specific assertions (harness-series.test.ts:103 and :115) use "withinExpectedTimeRate". TimePerPassedTaskChart calls pivotByHarness(data, harnesses, "timePerPassedTask") (wall-clock-chart.tsx:42-46), so the one metric that actually reaches the new chart has no pivot coverage. Give the factory a timePerPassedTask parameter and add one assertion mirroring line 103 for that key.
6. [Axis 3] Watchlist slow-task badge threshold moved from 1.5× to 2× with no view-level test (evalboard/app/watchlist/watchlist-view.tsx:546) — The red/amber gate changed value in this PR — r.avgTurnRatio > 1.5 became:

                                        className={`ml-auto font-semibold rounded-full px-2.5 py-0.5 text-[11px] border ${r.avgTimeRatio > 1 + TIME_BUDGET_TOLERANCE ? "bg-red-50 text-red-700 border-red-200" : "bg-amber-50 text-amber-700 border-amber-200"}`}

TIME_BUDGET_TOLERANCE is 1 (timing.ts:76), so the red cutoff is now 2×. app/watchlist/__tests__/watchlist-view.test.tsx was touched only to rename a fixture field (expectedTurns → expectedSeconds, line 38); its four tests (lines 56, 78, 102, 120) never render the renamed "🐌 Slow-task offenders" panel's badge. Add one test building a timeOverage row at 1.8× and one at 2.5× and assert bg-amber-50 vs bg-red-50, so the threshold is pinned rather than implicit.
7. [Axis 3] CE031's EXEMPT map gains its first entry, but nothing validates that an exemption names a real model or field (tests/lint/dead_config_fields.py:55) — The comment block promises a contract nothing checks:

# deliberate. Every entry must name a real field.
EXEMPT: dict[str, dict[str, str]] = {
    "RunLimits": {
        "expected_turns": (

find_dead_config_fields only reads EXEMPT.get(model.__name__, {}) (line ~87) and never asserts the exempted names exist in model.model_fields. tests/test_custom_lint.py::TestCE031DeadConfigFields (line 2038) has four tests — test_exempt_field_not_flagged (line 2083) uses a synthetic model, so the real map is unvalidated. When expected_turns is finally dropped from RunLimits ("dropped in a later minor"), the exemption silently lingers. Add one test: for every Model in EXEMPT, assert the model is in CONSUMED_MODELS and every exempted key is in model.model_fields.

What's Missing

Parallel paths:

• 🟠 src/coder_eval/orchestration/run_limits.py::validate_run_limits — the existing non-blocking run-limits warning seam, surfaced by coder-eval plan (plan_command.py:139) and by the orchestrator's _warn_on_ineffective_task_timeout — was not given an expected_turns is deprecated and ignored message, even though this PR deleted Orchestrator._check_expected_turns, the only code that ever produced output for the field. Pydantic's deprecated= fires on attribute access, and the CE031 exemption exists precisely because nothing reads the attribute any more, so loading a YAML that declares expected_turns (per the exemption text, ~930 of them) now emits nothing at all, on any surface, before the field is dropped in a later minor. (trigger: src/coder_eval/orchestrator.py)
• 🟠 evalboard/app/runs/[id]/task-grid.tsx renders every row twice — desktop <tbody> (654-657, cell at 705-708) and mobile card (794-797, Duration sub at 843-853) — each computing timeRatio + tintForTimeRatio + expectedTimeTitle inline rather than through one shared row helper. Every fix in this column therefore has to land twice (the missing matureSkipped guard, the title/MATURE_TOOLTIP swap, any threshold change), and only the desktop copy is reachable by the tests, whose cellFor deliberately scopes to screen.getByRole("table") (task-grid.test.tsx:41-44). (trigger: evalboard/app/runs/[id]/task-grid.tsx) (restates: Axis 6: mature-skipped zero-duration sentinel unguarded in the task-grid render paths)
• 🟡 The Python report surfaces lost the efficiency signal and gained no seconds-based replacement: reports.py::_runtime_notes_lines dropped the overage blockquote, reports_html.py::_render_header dropped the badge, reports_stats.py dropped expected_turns_overage, and nothing writes a duration-vs-expected note in their place. Wall-clock efficiency now exists only in the TypeScript dashboard, fed by an out-of-repo stamp — so a CLI-only or OSS user of coder-eval run has no efficiency signal on any surface, which the PR does not state. (trigger: src/coder_eval/reports.py) (restates: Axis 7: expected_seconds has no producer in this repo)
• 🟡 evalboard/lib/timing.ts declares itself a mirror of the runner's timing.py twice (header: "The dashboard reads that stamp rather than deriving its own, so a number here still matches the ping"; line 74: "Mirrors timing.TOLERANCE on the runner side") but ships no parity mechanism. The repo's own convention for a hand-copied cross-language mirror is a parity test — CLAUDE.md mandates it for pricing.py ⟷ pricing.ts, enforced by lib/__tests__/pricing-parity.test.ts. Here the runner's own time_per_passed_task and tolerance arrive in every run.json timing block, so the mirror is checkable at runtime, and nothing checks it. (trigger: evalboard/lib/timing.ts) (restates: Axis 1: the run.json timing block is parsed into RunOverview but unread)

Tests:

• 🟠 evalboard/app/trends/trends-view.tsx changed 100 lines to add the expected-time column (the new timeCellClasses/tintForTimeRatio/fmtTimeRatioCell cell at 294-320) and is the only surface that guards matureSkipped correctly, yet app/trends/__tests__/trends-view.test.tsx was not touched at all — its six tests cover the status strip and maturity badges only. The one implementation the run grid should be copied from has nothing pinning it. (trigger: evalboard/app/trends/trends-view.tsx)
• 🟠 evalboard/app/_overview/wall-clock-chart.tsx (TimePerPassedTaskChart, renamed from turn-budget-chart) has no test file — app/_overview/__tests__/ holds only harness-legend and harness-series. Its non-trivial new logic is the withinByPoint map keyed on the string join `${p.harness}|${p.timestamp}` (47-56) and read back with the same join in the tooltip secondary callback (95-105): a key-shape mismatch or a harness-normalization difference returns null and the within-expected line silently disappears from every hover, with nothing failing. _(trigger: evalboard/app/overview/wall-clock-chart.tsx)
• 🟡 Two exported helpers of the new lib/timing.ts have no direct unit test in lib/__tests__/timing.test.ts: timeCellClasses (tint → Tailwind class, including the null → untinted branch) and fmtTimeRatioCell (the one-decimal × chip added by the final commit, which drives task-grid, task-stats and trends). Both are only covered incidentally through component tests that assert rendered strings/classes, so a change to either is caught in three component suites or not at all. (trigger: evalboard/lib/tests/timing.test.ts)
• 🟡 The mobile card branch of the task grid — a full second rendering of Duration + the new ratio + tint (task-grid.tsx:794-797, 843-853) — has zero assertions; cellFor scopes queries to the desktop <table> by design, and the new vs Expected suite (task-grid.test.tsx:124-157) never queries the cards. Add card-scoped cases for at least the tinted, unscored and mature rows, or extract one shared row-model helper so the desktop tests cover both. (trigger: evalboard/app/runs/[id]/tests/task-grid.test.tsx)

Downstream consumers:

• 🟡 The same PR renamed TaskHistoryEntry.expectedTurns → expectedSeconds (lib/trends.ts:62, populated at :309), which changes the payload cached under ["aggregate-task-trends-v3"] (trends.ts:258) and ["history-for-task-v2"] (trends.ts:339) — neither key was bumped, despite both carrying a version token added by earlier, smaller shape changes with the explicit comment "the key bump keeps a stale pre-deploy payload from being served into new code". Applying the codebase's own convention to this PR means three key bumps, not one. (trigger: evalboard/lib/trends.ts) (restates: Axis 2: the per-run unstable_cache key was not versioned even though the cached PerRun shape changed)
• 🟡 The watchlist attention score swapped its input metric but kept the normalizer and weight unchanged: timeOverage = clamp01(mean(ratios) - 1) with TIME_WEIGHT = 20 (watchlist.ts:209-213) is the verbatim turn-ratio formula. A turn ratio is bounded by max_turns; a wall-clock ratio is not — one task_timeout breach at 600s against a 60s line is 10×, which alone drags a skill's mean past 2 and saturates the full 20-point segment. The scoring change is a formula change and neither the normalizer, the weight, nor a per-row cap was revisited for the new distribution. (trigger: evalboard/lib/watchlist.ts) (restates: Axis 8: watchlist time aggregates include failed/timed-out tasks with no outcome filter)
• 🟡 While expected_seconds is unstamped, the degradation is not uniformly "a blank column": in watchlist.ts::attention every timeRatio returns null, so ratios is empty, timeOverage is 0 for every skill, and 20 of the attention score's 100 points drop out — the ranking changes shape and the "🐌 Slow-task offenders" panel empties, rather than a cell reading —. The PR's forward-compatibility argument covers the em-dashed columns but not the reordered attention list a reviewer would read as a real improvement. (trigger: evalboard/lib/watchlist.ts) (restates: Axis 7: expected_seconds has no producer in this repo)

Display & mapping dicts:

• 🔵 COLUMN_HELP (task-grid.tsx:46-50) is Partial<Record<SortKey, string>> and gains only vsExp; duration and turns still have no entry, so those <th>s render title={undefined}. Both were redefined by this PR — lib/turns.ts now opens with "Turns are shown as a plain count on task views; they are no longer an efficiency signal", and Duration acquired a sibling ratio column — so the two columns whose meaning changed are the two with no help string at all. (trigger: evalboard/app/runs/[id]/task-grid.tsx)
• 🔵 Retired turn-budget vocabulary survives in surfaces this PR touched, including new test copy: app/_overview/__tests__/harness-legend.test.tsx still asserts suffix="within turn budget" / emptyText="no tasks with a turn budget"; app/page.tsx:222-226 still comments the analytics block as "daily success / turn-budget charts"; and lib/__tests__/trends.test.ts:174 is still named "propagates totalTurns and expectedTurns" for a field that no longer exists. _(trigger: evalboard/app/overview/tests/harness-legend.test.tsx) (restates: Axis 1: the turn-budget removal sweep left stale comments and retired vocabulary behind)
• 🔵 docs/tutorials/04-writing-a-task.md had its body rewritten (expected_turns → max_turns, plus the new efficiency paragraph) but not its framing: the frontmatter description (lines 2-5) still promises "prompt, isolated sandbox, turn budget, and three success criteria", and line 11 still says the reader will author "a soft turn budget". The frontmatter description is the published page/site blurb, so the tutorial advertises a concept its own body says was retired. (trigger: docs/tutorials/04-writing-a-task.md) (restates: Axis 1: the turn-budget removal sweep left stale comments and retired vocabulary behind)

Daily/nightly:

• 🟠 The PR states no nightly-pipeline plan for the reverse deploy skew it creates. eval_result_to_task_dict is the sole writer of run.json task rows and stops emitting expected_turns/expected_turns_overage the moment the first post-merge nightly runs, while the currently deployed board still reads them (runs.ts:738/:1045 on main) and gates turnBudgetRateForTasks on them — blanking the front-page chart, both Turns tint sites, and the watchlist turn-overage segment. This repo has no evalboard deploy workflow (.github/workflows/ has none), and the PR body files the fix under "Left to do" as a manual workflow_dispatch in another repo rather than as a merge prerequisite. (trigger: src/coder_eval/reports_experiment.py) (restates: Axis 8: dropping expected_turns from the run.json task_results[] contract blanks the deployed dashboard)
• 🟡 docs/REPORT_SCHEMA.md is the published cross-repo run.json contract and this PR leaves it stale in both directions: the diff removes expected_turns from the turn-accounting list (line 79) and documents neither the per-row expected_seconds nor the run-level timing block that the external eval_runner now stamps and that four evalboard surfaces read. The producer of the PR's only input metric therefore has no schema entry anywhere in this repo. (trigger: docs/REPORT_SCHEMA.md) (restates: Axis 7: expected_seconds has no producer in this repo)

Harness & Lint Improvements

Static checks (lint / type):

• [ce-lint] CE043 — evalboard dead-projection gate. Port CE031's dead-config idea to the TS read path: a new tests/lint/evalboard_dead_fields.py (whole-tree text/regex rule, wired as tests/test_custom_lint.py::TestCE043EvalboardDeadProjections — not a BaseRule, since the runner is Python-AST-only) that, for each field declared on the run.json projection types in evalboard/lib/runs.ts (RunOverview, RunOverviewTask, RawTiming, RawTaskResult), counts identifier occurrences across evalboard/**/*.{ts,tsx} excluding node_modules, the declaration itself, and __tests__/ fixtures. Zero non-test, non-write occurrences = a field computed per task/per run on the front page's hot read path that nothing reads; requires an EXEMPT entry with a reason. Pair it with knip (or ts-prune) added to make evalboard-verify for the exported-symbol half, which catches the orphan helper directly. Forbidden pattern in one line: a projection field or exported helper in evalboard/lib/ that only ever gets written. Prevents: Finding 1 (high, cross-axis 1/2/3/5/7/8): RunOverviewTask.visibleTurns (runs.ts:945) + the orphaned exported visibleTurnsFromRaw (runs.ts:987) + RunOverview.timePerPassedTask (runs.ts:978, written at :1098) + 7 of the 8 fields of the brand-new RawTiming interface (runs.ts:468-477), all with no reader anywhere.
• [ce-lint] CE044 — retired-vocabulary gate. A RETIRED: dict[str, str] map (dead symbol → what replaced it / the PR that removed it) in tests/lint/retired_vocabulary.py, scanned over src/, tasks/, experiments/, docs/, plugins/, .claude/ and evalboard/** (excluding node_modules, .next, and an explicit allowlist for CHANGELOG/migration notes). Any surviving mention — comment, docstring, YAML comment, test name, prose — fails with the map's replacement text. Rule of use: a symbol goes into the map only when it is fully removed; a deliberate deprecation shim (RunLimits.expected_turns) is carried as an allowlist entry instead, exactly like CE031's EXEMPT. Seed with expected_turns_overage, turnBudgetRate, openHelp, and the phrase "within expected turns". Prevents: Finding 4 (medium, axes 1/5/7): the orphaned 3-line comment in experiments/default.yaml:24-26 that now mis-annotates task_timeout as "does NOT abort the run" in the layer-1 defaults file every task inherits; the visibleTurnsFromRaw docstring at runs.ts:984-985 still citing the deleted "within expected turns" metric as its justification; and the task-grid.tsx comment describing the removed openHelp state.
• [ce-lint] CE045 — single pass predicate in evalboard. Forbid a raw "SUCCESS" string comparison (=== "SUCCESS", !== "SUCCESS", status === "SUCCESS" inside a filter) anywhere under evalboard/ except evalboard/lib/status.ts, which owns statusCategory / isPassStatus. Implement as an ESLint no-restricted-syntax selector once ESLint exists in evalboard (see CE047's note — the package has no linter today, only tsc, vitest, next build), or as a text rule alongside CE043 if the team prefers no new toolchain. The file already carries the convention as a prose comment at overview.ts:833-836; this makes it mechanical. Note the fix must be a lib-wide sweep, not a two-line edit — converting only the new call sites would introduce the asymmetry the finding warns about. Prevents: Finding 2 (medium, axes 1/2): overview.ts:59 and :81 — the numerator/denominator of both new headline time metrics — plus the pre-existing overview.ts:975, :1087, watchlist.ts:87, trends.ts:160 that the same sweep must convert.
• [ce-lint] CE046 — no hardcoded time-budget ratio in prose or defaults. Forbid the literals 2×, 1.5×, x expected, × expected and the bare numeric defaults 1.5 / 2 inside getTimeRatioThresholds() from appearing outside evalboard/lib/timing.ts; the single source is TIME_BUDGET_TOLERANCE, and UI strings must interpolate 1 + TIME_BUDGET_TOLERANCE. Also require the red tint default to derive from that constant (const RED_RATIO = 1 + TIME_BUDGET_TOLERANCE) rather than repeat 2, so the module comment's claim that "the cell and the rollup agree" becomes structurally true instead of true-by-coincidence-at-defaults. watchlist-view.tsx:546 already does the right thing and is the in-tree precedent. Prevents: Finding 3 (medium, axes 1/5/7): six non-derived copies of the boundary — timing.ts:25 (env-overridable red default), wall-clock-chart.tsx:104, task-grid.tsx:48, overview.ts:32 and :43 comments, harness-legend.test.tsx:66,74. Also finding 5 (low): the x expected clause forces the ASCII/×-glyph and 2-decimal/1-decimal formatter split at timing.ts:105 vs :112 to collapse to one formatter.
• [ce-lint] CE047 — no server-only env read reachable from a client component. Walk the evalboard import graph from every file whose first line is "use client"; flag any process.env.X read in that transitive closure where X does not start with NEXT_PUBLIC_. next.config.mjs carries no env passthrough, so such a read is undefined in the browser, silently falls back to its default, and produces a hydration mismatch because Next SSRs the same component on the server where the var resolves. The repo already states the rule in prose at run-view.tsx:174 ("process.env isn't readable in this client component") and lib/edition.ts is the worked precedent for threading the value down as a prop. This rule (and CE045/CE046/CE048's AST clauses) are the concrete argument for adding an ESLint config to evalboard/ and gating it in make evalboard-verify. Prevents: Finding (medium, axis 7): timing.ts:24-25 reading EVALBOARD_TIME_YELLOW_RATIO / EVALBOARD_TIME_RED_RATIO via tintForTimeRatio's default argument at three client call sites (task-grid.tsx:658, :798, trends-view.tsx:299) while the server-rendered task-stats.tsx:39 honors them — one setting, two different thresholds across the board. Vitest's vi.stubEnv tests pass regardless, so CI is green while the knob is inert.
• [ce-lint] CE048 — versioned unstable_cache keys. Require the first element of the keyParts array in every unstable_cache(...) call under evalboard/ to match -v\d+$. Next's key is hash([MAIN_KEY_PREFIX, fetchCacheKeyPrefix, cb.toString() + keyParts]) with no build id participating, so an unversioned key is byte-identical before and after a payload-shape change. The codebase already follows the convention at trends.ts:258 (aggregate-task-trends-v3) and :339 (history-for-task-v2) — this makes it enforced rather than remembered. This rule catches the missing token; it cannot know a shape changed, which is what the paired harness snapshot (below) is for. Prevents: Finding (medium, axis 2): overview.ts:292 ["evalboard-per-run", source.id] never versioned despite expectedTurns→expectedSeconds and +timePerPassedTask. The same rule flags that aggregate-task-trends-v3 and history-for-task-v2 also needed bumps this PR (trends.ts:62 changed TaskHistoryEntry), i.e. three bumps, not one.
• [ce-lint] CE049 — run.json task-row contract parity. Extend tests/lint/doc_schema_parity.py (CE030's engine) with a second direction: AST-read the key set of the dict literal returned by reports_experiment.eval_result_to_task_dict (the sole writer of run.json task rows, per orchestration/batch.py:673) and require every key to appear as inline code in docs/REPORT_SCHEMA.md, and vice versa. A key that disappears from the producer without disappearing from the doc — or a doc that quietly drops a concept — fails the build, which is the signal a deployed consumer needs. Prevents: Finding (high, axis 8): reports_experiment.py:192 silently dropping "expected_turns" and "expected_turns_overage" from every new run.json, blanking four surfaces of the currently-deployed dashboard with no release note. Also finding (high, axis 7): docs/REPORT_SCHEMA.md losing expected_turns from the turn-accounting list while gaining no expected_seconds entry.
• [ce-lint] CE050 — externally-stamped fields must be declared as such. A companion map EXTERNALLY_STAMPED: dict[str, str] (run.json field → producing repo/module) in the same doc-parity module. Any run.json field documented in docs/** as "derived"/"stamped"/"computed" that has no writer under src/ must be listed there, and each doc page mentioning it must carry the producer's name (grep the surrounding paragraph for the map's value). This turns "the eval runner does it" into a checked statement about a component that is not part of coder-eval. Prevents: Finding (high, axis 7): grep -rn 'expected_seconds' src/ tests/ returns nothing, yet docs/TASK_DEFINITION_GUIDE.md:316-321 and docs/tutorials/04-writing-a-task.md:66-67 present the stamp as automatic, and timing.ts:120 tells OSS users "no expected time yet (needs a passing run on this harness)" — a promise a standalone coder-eval run can never fulfil, on an ungated column (task-grid.tsx:345) that ships permanently em-dashed.
• [ce-lint] Extend CE031 to validate its own EXEMPT map. Add one test to TestCE031DeadConfigFields: for every key in tests/lint/dead_config_fields.py::EXEMPT, assert the model is in CONSUMED_MODELS and every exempted field name is in model.model_fields. The module comment already promises "Every entry must name a real field", but find_dead_config_fields only does EXEMPT.get(model.__name__, {}) and never checks the names exist; the four existing tests use synthetic models, so the real map is unvalidated. Same shape as the runner's duplicate-rule-id assertion — a self-check on the lint config, not on src/. Prevents: Finding (low, axis 3): the map's first-ever entry (RunLimits.expected_turns, exempted as "dropped in a later minor") would silently linger as a stale exemption once the field is actually removed, re-opening the dead-config hole CE031 exists to close.
• [ce-lint] ESLint baseline for evalboard/ (the package has no linter at all today). Beyond the rules above, three off-the-shelf/near-off-the-shelf settings, wired into pnpm verify so make evalboard-verify gates them: (a) @typescript-eslint/no-non-null-assertion — the ! operators leaning on an invariant that lives in another module; (b) a no-restricted-syntax selector banning ?? -Infinity (or any non-finite sentinel) as an operand of - inside a comparator, since -Infinity - -Infinity is NaN and NaN !== 0 swallows the tie-break; (c) a selector banning expectedSeconds != null outside lib/timing.ts, forcing one exported hasExpectedTime(s): s is number predicate so the module header's claim ("Every helper returns null there") holds. Plus eslint-plugin-jsx-a11y with a project rule banning title= as the only help affordance on non-focusable JSX (th, h1–h6, bare span). Prevents: Finding (low, axis 2) watchlist.ts:343-344 t.durationSeconds! / t.expectedSeconds!; finding (low, axis 2) task-grid.tsx:303's vsExp comparator returning NaN for every unscored pair — i.e. every historical run — leaving the grid in raw run.json order; finding (low, axis 2) the two disagreeing "unscored" predicates at timing.ts:117 and task-stats.tsx:66 that render 0m00s for expected_seconds: 0; and finding (medium, axis 7) the help demoted to title on a non-interactive <th> (task-grid.tsx:626) and <h2> (page.tsx:288) after HelpPopover/ColHelpIcon were deleted.
• [pyright] Type-level exhaustiveness for the harness-metric test factory (the TypeScript half of the same "let the type checker enumerate the cases" gate — tsc --noEmit in make evalboard-verify; this schema has no tsc kind). Change the harness-series.test.ts row factory to build its metrics from a Record<HarnessMetricKey, number | null> derived from keyof HarnessMetric, so widening the metric union fails compilation until the new member is supplied and asserted, instead of being quietly defaulted. Same pattern applies to any future pivot-style union. Prevents: Finding (low, axis 3): HarnessMetric widened to three members (harness-series.ts:15-18), but the factory hardcodes timePerPassedTask: null and both metric assertions (:103, :115) use withinExpectedTimeRate — so the only metric the new wall-clock chart plots (wall-clock-chart.tsx:42-46) has zero pivot coverage.

Harness improvements (not statically reachable):

• Producer-generated run.json fixture as the evalboard test corpus. Add a make evalboard-fixture target that runs the Python producer (eval_result_to_task_dict over a canned EvaluationResult) and writes evalboard/lib/__tests__/fixtures/run.json, committed and re-generated + diffed in CI. Every evalboard reader test then parses the real producer output instead of a hand-written literal, so a key removed on the Python side breaks the JS suite in the same PR. Why not static: Producer and consumer are in different languages and (for the deploy) different repos; the coupling is data-shaped — knowing the emitted key set for a realistic result requires actually executing the serializer (defaults, None-elision, nested dicts), which no single-tree lint can compute. CE049 checks the producer against the docs; only an executed fixture checks it against the consumer. Prevents: Finding (high, axis 8) — expected_turns / expected_turns_overage removed at reports_experiment.py:192 with every evalboard test still green because each fixture is hand-authored.
• Backward-skew CI job. Build evalboard from origin/main (i.e. what is deployed) and feed it a run.json produced by the PR HEAD; assert the headline metric functions (turnBudgetRateForTasks / withinExpectedTimeRateForTasks / timePerPassedTaskForTasks) return non-null for at least one run and that no chart series is empty. Today only the forward direction (new board, old run.json) was verified — the reverse is the one that actually happens on merge. Why not static: Requires two builds of the dashboard plus executing the metric code against real rows; "old consumer + new producer" is a cross-commit property, invisible to any check that sees one tree. Prevents: Finding (high, axis 8) — the front-page "Within Expected Turns (%)" chart plotting nothing, run-page and mobile Turns cells losing their tint, and the watchlist turn-overage segment going empty, from the first post-merge nightly until someone remembers the manual workflow_dispatch deploy.
• Cached-payload shape snapshot bound to the cache key. One vitest snapshot per unstable_cache call recording the literal key string plus the sorted field set of the payload type (via a satisfies-driven key list or a small ts-morph extraction). Any field added/removed/renamed fails the snapshot, and the fix message says: bump -vN and re-snapshot. Why not static: The invariant is "shape changed ⇒ key string must change", which needs a stored baseline of the previously-shipped shape. A single-tree lint (CE048) can only require that a version token exists, never that it was incremented for this change. Prevents: Finding (medium, axis 2) — evalboard-per-run serving a pre-deploy PerRun into new code for up to 300 s, blanking the wall-clock chart, /trends expected-time cells and the watchlist time rail; plus the two unbumped sibling keys (aggregate-task-trends-v3, history-for-task-v2) whose TaskHistoryEntry also changed at trends.ts:62.
• Shared pathological-row fixture set + a surface checklist test. One exported fixture module with the four rows every time surface must handle — carried-forward (matureSkipped: true, durationSeconds: 0, expectedSeconds: 100), unscored (expectedSeconds: null), degenerate (expectedSeconds: 0), and failed/timed-out (status: "TASK_TIMEOUT", durationSeconds: 600) — plus a checklist test that enumerates every module importing timeRatio and fails when one has no case for each row. The repo already documents the 0-duration contract twice (runs.ts:948-950, trends.ts:162-164) and the PR's own fixture encodes it (overview.test.ts:211-222), so the fixture is codifying an existing written rule. Why not static: The correct handling differs per surface — — in a cell, excluded from an aggregate, or legitimately included in a numerator (timePerPassedTaskForTasks keeps failure seconds on purpose) — so it is semantic per call site, only observable by rendering/aggregating. A lint can flag an unguarded timeRatio(t.durationSeconds, …) shape, but not which of the three treatments is right. Prevents: Finding (high, axis 6) — a carried-forward task rendering a green 0.0× "beat its expected time by 100×" at task-grid.tsx:654 and :794 while trends-view.tsx and both aggregates guard correctly; finding (medium, axis 6) — watchlist.ts:194 and :340 folding unbounded timeout ratios (600 s/60 s = 10×) into a panel titled "Passing, but well past their expected time", double-charging the same failure the fail-rate segment already scored; finding (low, axis 2) — expected_seconds: 0 rendering — in the cell and 0m00s in the tooltip on the same element.
• Client-bundle env audit after next build. A CI step that greps the emitted .next client chunks for each EVALBOARD_* name the code reads, asserting that non-NEXT_PUBLIC_ names appear in server chunks only and that any knob intended for the browser is NEXT_PUBLIC_-prefixed and actually inlined. Pair with documenting both threshold vars in evalboard/README.md, and consider extending CE027's parity to the reverse direction for the EVALBOARD_ prefix it currently excludes (a var read in evalboard/ must be documented there). Why not static: Whether a value survives into the client bundle is a property of the bundler's output for the current next.config.mjs, not of the source tree; CE047 catches the source-level shape, this proves the deployed artifact. Prevents: Finding (medium, axis 7) — EVALBOARD_TIME_YELLOW_RATIO / EVALBOARD_TIME_RED_RATIO inert in the browser, undocumented anywhere, and green in CI because vi.stubEnv exercises Node's process.env rather than the bundle.
• Make the dashboard deploy part of merging a contract change. Either add a repository_dispatch/workflow_call deploy that fires on merges touching the run.json contract, or a pr-checks job that fails when a PR changes the keys emitted by eval_result_to_task_dict and evalboard/lib/runs.ts unless the PR body carries an explicit Deploy: <workflow> line. Today .github/workflows/ here has no evalboard deploy at all and the required dispatch lives only in the PR's "Left to do" notes. Why not static: Deployment ordering is a process/infrastructure property outside the repository tree; no tree-local check can know whether the consumer has been redeployed. Prevents: Finding (high, axis 8) — the exposure window between merge and a remembered manual dispatch, during which every new run.json degrades three-to-four dashboard surfaces with no error and no gap indicator.
• Threshold- and a11y-pinning tests for shared-constant UI gates. (a) Render a timeOverage row at 1.8× and one at 2.5× and assert bg-amber-50 vs bg-red-50, so the badge cutoff is pinned to 1 + TIME_BUDGET_TOLERANCE rather than implicit. (b) A keyboard/AT reachability test (jest-axe or a focus-order assertion) for column and chart help, asserting the definition is reachable without a pointer hover. Why not static: A tint/threshold boundary is a rendered-output property of a value that lint sees only as a number, and assistive-technology reachability is a runtime focus/announcement property — ESLint can ban the title-only anti-pattern shape but cannot prove the replacement help is actually reachable. Prevents: Finding (low, axis 3) — the watchlist slow-task badge moving from 1.5× to 2× with all four existing tests never rendering that panel; finding (medium, axis 7) — the front-page metric definition and the per-column help becoming hover-only after HelpPopover/ColHelpIcon were deleted, unreachable on touch and by keyboard.

Top 5 Priority Actions

1. Guard the carried-forward sentinel in both vs Expected render paths — evalboard/app/runs/[id]/task-grid.tsx:654 (desktop) and :794 (mobile) call timeRatio(t.durationSeconds, t.expectedSeconds) with no matureSkipped check, so a task that never executed renders a green 0.0× beside a real expected time; mirror trends-view.tsx (t.matureSkipped ? null : timeRatio(...) plus MATURE_TOOLTIP) and pin it with a vsExpCellFor test.
2. Decide the deploy order for the dropped run.json contract at src/coder_eval/reports_experiment.py:192 — removing expected_turns/expected_turns_overage blanks the currently-deployed board's headline chart, both Turns tint sites and the watchlist turn-overage rail with no error, and the evalboard deploy is a manual workflow_dispatch, so either keep emitting expected_turns for one release or make the dispatch a stated merge step.
3. Add the pass-only filter the headline already uses to the two watchlist time aggregates — evalboard/lib/watchlist.ts:194 and :340 push every ratio regardless of outcome, and because a task_timeout inflates the time ratio without bound (unlike the old turn ratio, which a crash deflated), the "🐌 Slow-task offenders" panel can list an all-timeout skill under "Passing, but well past their expected time" while double-charging it the full 20-point time segment.
4. Stop presenting expected_seconds as automatic — nothing in src/ writes it (only the out-of-repo eval_runner), yet docs/TASK_DEFINITION_GUIDE.md:314, docs/tutorials/04-writing-a-task.md:66 and the tooltip at evalboard/lib/timing.ts:120 ("needs a passing run on this harness") promise a stamp an OSS coder-eval run can never produce, and the ungated vs Expected column ships to OSS permanently em-dashed; name the external producer in both docs pages plus docs/REPORT_SCHEMA.md and reword the empty state.
5. Finish the data-layer plumbing for the new metric: delete visibleTurns/visibleTurnsFromRaw and the inert RawTiming (7 of 8 fields have no reader) or wire timePerPassedTask/tolerance into overview.ts as the comments at evalboard/lib/runs.ts:465, :977 and :984 claim, and bump the three unversioned unstable_cache keys whose payload shape changed (overview.ts:292, trends.ts:258, trends.ts:339).

---

Stats: 0 🔴 · 4 🟠 · 7 🟡 · 7 🔵 across 8 axes reviewed.