perf(bench): exclude resolution fixtures from incremental-benchmark sweep#1131
perf(bench): exclude resolution fixtures from incremental-benchmark sweep#1131carlos-alm wants to merge 1 commit into
Conversation
…weep The incremental benchmark walks the repo root, which pulls hand-annotated resolution-benchmark fixtures into the corpus. A single heavy grammar (tree-sitter-verilog #1107) added ~850ms to fullBuildMs — the cost is real but unrelated to shared code paths the benchmark is meant to track. Add an optional exclude glob array to BuildGraphOpts, merged into config.exclude in setupPipeline so both the JS pipeline and the native orchestrator (which receives ctx.config as JSON) honor it. Pass 'tests/benchmarks/resolution/fixtures/**' from the benchmark script so future language PRs don't silently inflate timings, and remove the 3.10.0:Full build entry from KNOWN_REGRESSIONS now that the cause is addressed. Closes #1112
Greptile SummaryThis PR adds an optional
Confidence Score: 5/5Safe to merge — the change is narrowly scoped to benchmark infrastructure and a non-breaking additive field on BuildGraphOpts. The exclude merging in pipeline.ts is applied before initializeEngine, so both execution paths see the updated list. CodegraphConfig.exclude is typed as a required string[] and always initialised from DEFAULTS, so the spread is safe. The config cache returns structuredClone'd copies, so benchmark runs don't pollute the cache. All buildGraph call sites in the benchmark are updated consistently via BUILD_OPTS. The KNOWN_REGRESSIONS cleanup is backed by local smoke-test evidence and aligns with the fix. No files require special attention. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["buildGraph(root, opts)"] --> B["setupPipeline(ctx)"]
B --> C["ctx.config = loadConfig(rootDir)"]
C --> D{"opts.exclude\npresent?"}
D -- Yes --> E["ctx.config.exclude = [...config.exclude, ...opts.exclude]"]
D -- No --> F["ctx.config unchanged"]
E --> G["initializeEngine(ctx)"]
F --> G
G --> H{"native\norchestrator?"}
H -- Yes --> I["Rust orchestrator\nreceives ctx.config as JSON\n(honours merged exclude)"]
H -- No --> J["JS pipeline\n(honours merged exclude)"]
I --> K["File collection skips\nexcluded globs"]
J --> K
subgraph "incremental-benchmark.ts"
L["BENCH_EXCLUDE = ['tests/benchmarks/resolution/fixtures/**']"]
M["BUILD_OPTS = { engine, exclude: BENCH_EXCLUDE }"]
N["buildGraph(root, { ...BUILD_OPTS, incremental: false })"]
O["buildGraph(root, { ...BUILD_OPTS, incremental: true })"]
L --> M
M --> N
M --> O
end
Reviews (1): Last reviewed commit: "perf(bench): exclude resolution fixtures..." | Re-trigger Greptile |
Summary
excludeglob array toBuildGraphOpts, merged intoconfig.excludeinsetupPipelineso both the JS pipeline and the native orchestrator (which receivesctx.configas JSON) honor it.tests/benchmarks/resolution/fixtures/**fromscripts/incremental-benchmark.tsso future language PRs don't silently inflatefullBuildMspurely from fixture parsing (e.g. tree-sitter-verilog added ~850ms / +43% on1959 → 2809msin feat(native): port Verilog extractor to Rust #1107).3.10.0:Full buildentry (and its docstring paragraph) fromKNOWN_REGRESSIONSnow that the root cause is addressed.Smoke-tested locally against this worktree (WASM engine):
Closes #1112
Test plan
npm run typecheckcleannpm run lintclean on changed filesRUN_REGRESSION_GUARD=1 npm test -- tests/benchmarks/regression-guard.test.ts→ 17/17 pass after removing the exemption (committed3.10.0entry already had a clean 1959ms baseline; only the rollingdevcomparison needed the exemption while fixtures inflated the number)exclude: ['tests/benchmarks/resolution/fixtures/**'], no fixture nodes land in the DB; without it, 25 verilog nodes are presentdev → 3.10.0Full build comparison no longer hits the threshold