Skip to content

feat(bench): add --out summary CSV flag + verify v0.5.35 CLI hot-path baseline#247

Merged
githubrobbi merged 1 commit into
mainfrom
docs/close-phase-6-24h-rerun-2026-05-15
May 15, 2026
Merged

feat(bench): add --out summary CSV flag + verify v0.5.35 CLI hot-path baseline#247
githubrobbi merged 1 commit into
mainfrom
docs/close-phase-6-24h-rerun-2026-05-15

Conversation

@githubrobbi
Copy link
Copy Markdown
Collaborator

@githubrobbi githubrobbi commented May 15, 2026

Standalone v0.6.0 release-readiness work, surfaced during the 2026-05-15 baseline-bench pass on the Windows 7-drive reference box.

Note on rebase. This PR originally also carried the Phase 6 24-h soak closure commit, but that landed independently on main as PR #246 while this branch was being prepared. Rebasing origin/main auto-detected the duplicate patch and dropped it; only the bench commit remains. See PR #246 for the Phase 6 soak closure details.

Plan §1 goal-4 (no CLI hot-path regression) — VERIFIED

Captured 2026-05-15 on the 7-drive reference box (drive D,
7.07 M records, 30 rounds, HOT phase, p50/p95 wall_ms):

Pattern v0.5.35 v0.5.96 Δ p50 Rows
exact 20 / 23 ms 18 / 19 ms −10 % 3
prefix 46 / 50 ms 40 / 46 ms −13 % 8 732
ext_rare 18 / 20 ms 17 / 18 ms −6 % 11
ext_dll 94 / 100 ms 35 / 40 ms −63 % (2.7×) 44 529
substring 50 / 54 ms 43 / 47 ms −14 % 12 458

Current v0.5.96 is universally faster than v0.5.35 on every
pattern. Row-count parity preserved exactly across both versions
(no result-set drift). Capture: LOG/memory output (manual
stdout paste — the bench script's --out flag was not yet
available for this run).

New bench-script feature: --out <path>

scripts/windows/cross-tool-benchmark.rs learned a --out <path>
flag that writes the post-run summary as plain CSV (12 columns:
tool,phase,sink,drive,pattern,p50_ms,p95_ms,rows,bad,verdict, rounds_ok,rounds_total). Intermediate directories are created
on demand; the writer exits 2 on failure. Distinct from the
existing <cwd>/uffs_bench_out.csv daemon-side file (raw
per-query row dumps). Banner block now prints both paths up
front so operators see exactly where each artefact lands.

Unknown CLI flags now warn on stderr (warning: unknown flag "--foo" ignored) instead of silently swallowing — caught during
the 2026-05-15 baseline bench where a typo---out in
pre-flag-support runs left the operator hunting a non-existent
CSV file.

v0.6.0 bake gate status (with this PR)

Gate Status Closed
Phase 5 (ws-trace) 24-h soak 🟢 4 / 4 PASS 2026-05-13
Phase 7 (USN-journal churn) 24-h soak 🟢 7 / 7 PASS 2026-05-13
Phase 6 (adaptive TTL) 24-h soak 🟢 9 / 9 PASS (#246) 2026-05-15
Plan §1 goal-4 (CLI hot-path no-regression) 🟢 5/5 patterns improved 2026-05-15

All v0.6.0 readiness gates green. Only the one-week main bake
remains per
memory-tiering-bake-criteria.md.

Verification

  • just lint-pre-push — 23/23 gates green (54s pre-commit; 50s
    pre-push initial; 3s pre-push post-rebase — bucket 2 skipped
    since no rust/dep/infra files changed).
  • Bench script compiles cleanly via standalone rustc --edition 2021 --emit=metadata on Mac (frontmatter intact, no warnings).
  • No new code paths in production crates — this is documentation
    plus a Windows-only rust-script.

Files touched

  • CHANGELOG.md — two new subsections under v0.6.0 staging:
    ### Verified — v0.5.35 baseline CLI hot-path bench (2026-05-15)
    and ### Added — cross-tool benchmark script: --out summary CSV flag.
  • scripts/windows/cross-tool-benchmark.rs--out flag
    plumbing + write_summary_csv impl (~80 LOC of new code) +
    unknown-flag warning + banner-block restructuring.

@githubrobbi githubrobbi enabled auto-merge (squash) May 15, 2026 13:46
… baseline

Two related additions for v0.6.0 release readiness, both surfaced
during the 2026-05-15 baseline-bench pass on the Windows 7-drive
reference box.

## scripts/windows/cross-tool-benchmark.rs

Add a `--out <path>` flag that writes the post-run summary as a
plain-CSV alongside the existing stdout tables.  Twelve-column
schema (`tool,phase,sink,drive,pattern,p50_ms,p95_ms,rows,bad,
verdict,rounds_ok,rounds_total`), integer milliseconds + integer
row counts, intermediate directories created on demand, exits 2
on write failure.  Distinct from the existing
`<cwd>/uffs_bench_out.csv` daemon-side file (raw per-query rows,
overwritten every round); the banner block prints both paths up
front so operators see where each artefact lands.

Unknown CLI flags now warn on stderr (`warning: unknown flag
"--foo" ignored`) instead of silently swallowing — caught during
the same 2026-05-15 bench where a typo-`--out` in pre-flag-support
runs left the operator hunting a non-existent CSV file.

## CHANGELOG.md

- New `Verified — v0.5.35 baseline CLI hot-path bench` subsection
  records plan §1 goal-4 PASSED: current v0.5.96 is universally
  faster than v0.5.35 across all 5 patterns (drive D, 7.07 M
  records, 30 rounds, HOT phase), with `ext_dll` (44 529 rows)
  showing a 2.7× speedup (94→35 ms p50).  Row-count parity
  preserved exactly across both versions on every pattern.
  Capture: `LOG/memory output` (manual paste).

- New `Added — cross-tool benchmark script: --out summary CSV`
  subsection documents the new flag, CSV column schema, and the
  unknown-flag-warns improvement.

## Verification

- `just lint-pre-push` — 23/23 gates green (54s).
- Bench script compiles cleanly via standalone `rustc --edition
  2021 --emit=metadata` on Mac (frontmatter intact, no warnings).
@githubrobbi githubrobbi force-pushed the docs/close-phase-6-24h-rerun-2026-05-15 branch from 5c50e10 to 8c6bd59 Compare May 15, 2026 13:47
@githubrobbi githubrobbi changed the title feat(release): v0.6.0 readiness — Phase 6 24-h soak closure + v0.5.35 bench baseline + cross-tool-benchmark --out flag feat(bench): add --out summary CSV flag + verify v0.5.35 CLI hot-path baseline May 15, 2026
@githubrobbi githubrobbi merged commit 2e1a8d7 into main May 15, 2026
27 checks passed
@githubrobbi githubrobbi deleted the docs/close-phase-6-24h-rerun-2026-05-15 branch May 15, 2026 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant