Local OpenROAD-SYN QoR signal: forced-engine variants, fast synth tier, asap7 suites, syn-dashboard

[oharboe]

The local counterpart of the Jenkins QoR signal (https://dashboard.precisioninno.com/), built through Bazel so the OpenROAD-SYN loop can be sharpened onto exactly the design, engine, and stage being worked on — fast enough for an agent (e.g. Claude) to iterate against. Supersedes #4363, which bundled status graphs and opt-in LEC (closed, parked there for revival).

What this adds

Every design gets manual forced-engine flow variants from the config.mk DSL, regardless of the engine its config.mk selects: <name>_syn_* pins SYNTH_USE_SYN=1 so any design can be tested under OpenROAD-SYN before its config.mk officially switches, and <name>_yosys_* pins yosys so the baseline stays reproducible after a switch (asap7/gcd today). gcd's single-process comparison now consumes the generated gcd_yosys_* variant instead of hand-rolling its own yosys flow.

Designs with a rules-base.json get two QoR tiers per variant, both checking the design's regular rules file with the regular checkMetadata.py — congruent with what Jenkins measures because the fast tier is a subset of the same gate, never a different metric:

• <name>_<engine>_synth_test — synthesis only, gated on the synth__/constraints__ subset of the rules (make metadata-check-synth). Minutes. QoR-only: without LEC it cannot prove correctness, so it's for iterating, not concluding.
• <name>_<engine>_test — the full flow against all rules: the real, dashboard-equivalent gate. It reuses the fast tier's synthesis action, so escalating from fast to full never repeats synthesis.

bazelisk test //flow/designs/asap7/jpeg:jpeg_encoder_syn_synth_test   # fastest signal, one design
bazelisk test //flow/designs/asap7/jpeg:jpeg_encoder_syn_test         # the real gate
bazelisk build //flow/designs/asap7/jpeg:jpeg_encoder_syn_route       # stage-level triage
bazelisk test --keep_going //flow/designs/asap7:syn_synth_test        # fast suite (14 designs)
bazelisk test --keep_going //flow/designs/asap7:syn_test              # full-flow suite

Displaying what you have is a separate concern from deciding what to build. bazelisk run //flow/designs:syn-dashboard prints a GitHub-markdown TL;DR over every forced-engine result already in bazel-testlogs — pass/fail per tier and engine, failing rule fields, result age — building nothing, pasteable into a PR as-is. See flow/designs/README.md for the loop.

The suites deliberately include designs that fail OpenROAD-SYN today — they are a status signal, not a green-only gate (see SYN_TEST_DESIGNS / SYN_SLOW_DESIGNS in flow/designs/asap7/BUILD).

Mechanism

• bazel-orfs bump picks up Syn fast qor bazel-orfs#771 (no patches carried): SYNTH_USE_SYN=1 engine selection with logs/1_synth.json declared (without it, no Bazel test could gate synth__* metrics), the generic fast synthesis-stage QoR pre-check (its own <variant>_synth sub-variant sharing the flow's synth action), the orfs_update cp fix, and the orfs_design(extra=) hook the DSL uses.
• ORFS side: metadata-check-synth / metadata-synth make targets (checkMetadata.py --only-prefix), genMetrics.py tolerance for synthesis-only trees, synth_syn.tcl always writing the 1_synth.v netlist (write_lec_verilog), the forced-engine variants in flow/designs/design.bzl, the two asap7 suites, and the dashboard.

Verified

Full flow under forced SYN and forced yosys on gcd (plus the regular gcd_test), fast tier green on uart and aes, aes-block failing its area rule honestly (synth__design__instance__area__stdcell), mock-alu failing to build (slang rejects). Sample dashboard output:

design	SYN synth	SYN flow	yosys synth	yosys flow	failing rules	last run
asap7/aes	✅	—	—	—		1h ago
asap7/aes-block	❌	—	—	—	synth__design__instance__area__stdcell	1h ago
asap7/gcd	✅	✅	✅	✅		just now
asap7/uart	✅	—	—	—		1h ago

Full-suite syn_test status across all 14 designs will be reported here once the multi-hour run completes.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces a local OpenROAD-SYN testing loop, including a local status dashboard (syn-dashboard), test suites for asap7, and support for synthesis-only QoR checks. Feedback on the changes highlights three main areas for improvement: correcting the output path of write_lec_verilog in synth_syn.tcl to prevent Bazel build failures, executing metadata-synth prerequisites sequentially in utils.mk to avoid parallel make race conditions, and explicitly specifying UTF-8 encoding when reading log files in synDashboard.py to ensure cross-platform compatibility.

[oharboe]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a fast local OpenROAD-SYN testing loop, including new test suites for asap7 designs, synthesis-only metadata checks, and a local markdown dashboard utility (synDashboard.py) to display test results. Feedback on the new dashboard script suggests making the XML parsing more robust against malformed attributes and renaming a generator variable to avoid shadowing an outer scope variable.

[oharboe]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a fast local OpenROAD-SYN testing loop and a local QoR dashboard. It adds forced-engine flow variants (syn and yosys) for designs, defines test suites for asap7, and implements a synDashboard tool to display test results from bazel-testlogs. Additionally, it updates metadata checking and metrics generation to support synthesis-only runs. Feedback on the changes points out that the generated gate-level netlist in synth_syn.tcl is written to the current working directory instead of the designated results directory, which will cause Bazel builds to fail.

[oharboe]

Full asap7 suite status (first complete run of both tiers)

bazelisk test --keep_going //flow/designs/asap7:syn_synth_test → 10/14 pass (mock-alu, mock-cpu, riscv32i fail to build — slang rejects; aes-block fails the area gate).

bazelisk test --keep_going //flow/designs/asap7:syn_test → 5/14 pass. This is the first time the synth-green designs meet the full-flow gates under OpenROAD-SYN, and it's the headline finding: synthesis QoR parity does not yet translate to flow QoR parity — aes, aes-mbff, ethmac, ethmac_lvt and uart all synthesize within their yosys-derived synth gates but fail timing from CTS onward.

bazelisk run //flow/designs:syn-dashboard (pasted as-is):

design	SYN synth	SYN flow	yosys synth	yosys flow	failing rules	last run
asap7/aes	✅	❌	—	—	cts__timing__setup__tns	15m ago
asap7/aes-block	❌	❌	—	—	synth__design__instance__area__stdcell, placeopt__design__instance__count__stdcell, detailedroute__route__wirelength	17m ago
asap7/aes-mbff	✅	❌	—	—	cts__timing__setup__tns, globalroute__timing__setup__tns	15m ago
asap7/aes_lvt	✅	✅	—	—		17m ago
asap7/ethmac	✅	❌	—	—	cts__timing__setup__ws, cts__timing__setup__tns, globalroute__timing__setup__ws, +3 more	just now
asap7/ethmac_lvt	✅	❌	—	—	cts__timing__setup__ws, cts__timing__setup__tns, globalroute__timing__setup__ws, +3 more	just now
asap7/gcd	✅	✅	✅	✅		48m ago
asap7/gcd-ccs	✅	✅	—	—		19m ago
asap7/jpeg	✅	✅	—	—		3m ago
asap7/jpeg_lvt	✅	✅	—	—		3m ago
asap7/uart	✅	❌	—	—	cts__timing__setup__ws, cts__timing__setup__tns, globalroute__timing__setup__ws, +3 more	22m ago

(mock-alu, mock-cpu, riscv32i are absent: their synthesis fails to build, so bazel records no test result — the FAILED TO BUILD entries in the suite output are the signal for those.)

Full-flow pass: gcd, gcd-ccs, aes_lvt, jpeg, jpeg_lvt. Interesting pair: aes fails cts__timing__setup__tns while aes_lvt (same RTL, LVT libs) passes — and jpeg passes end-to-end while ethmac misses timing broadly. These per-design, per-stage rule failures are exactly the loop targets this PR is meant to expose.

🤖 Generated with Claude Code

[oharboe]

Small consumability note from testing this PR's head as a bazel-orfs git_override pin: flow/designs/gf180/aes fails to load — its BUILD passes a DESIGN_TYPE argument that isn't registered in flow/scripts/variables.yaml, so bazel-orfs's check_variables errors the package and any @orfs//... wildcard along with it (Unknown ORFS variable(s) in arguments: DESIGN_TYPE). Every package we actually consume loads clean; registering the variable (or dropping the argument) would make wildcards usable again.