Local OpenROAD-SYN QoR signal: forced-engine variants, fast synth tier, asap7 suites, syn-dashboard

MODULE.bazel

@@ -101,7 +101,7 @@ bazel_dep(name = "bazel-orfs", version = "0.0.1")
 
 bazel_dep(name = "bazel-orfs-verilog", dev_dependency = True)
 
-BAZEL_ORFS_COMMIT = "6ebadeb4be5c9ada103081c9a5e668c014126616"
+BAZEL_ORFS_COMMIT = "3342596c1f7017b9fda5b4a55a25a54ff2aaedf8"
 
 BAZEL_ORFS_REMOTE = "https://github.com/The-OpenROAD-Project/bazel-orfs.git"
 

flow/designs/BUILD

@@ -1 +1,12 @@
 exports_files(["BUILD"])
+
+# Local TL;DR of https://dashboard.precisioninno.com/: a GitHub-markdown
+# status table over the forced-engine OpenROAD-SYN QoR tests already
+# built in this workspace (see flow/designs/design.bzl). Reads
+# bazel-testlogs only — builds nothing, instant.
+#
+#   bazelisk run //flow/designs:syn-dashboard
+alias(
+    name = "syn-dashboard",
+    actual = "//flow/util:synDashboard",
+)

flow/designs/README.md

@@ -0,0 +1,72 @@
+# Fast local OpenROAD-SYN testing loop
+
+The maintainers' QoR signal comes from Jenkins runs published at
+<https://dashboard.precisioninno.com/>. The targets described here are
+its local counterpart: the same designs, gated by the same
+`rules-base.json` files with the same `checkMetadata.py`, but built
+locally through Bazel so the loop can be sharpened onto exactly the
+design, engine, and stage being worked on — fast enough for an agent
+(e.g. Claude) to iterate against.
+
+Every design gets manual forced-engine flow variants (see
+`design.bzl`), regardless of the engine its config.mk selects:
+
+- `<name>_syn_*` pins `SYNTH_USE_SYN=1` (OpenROAD-SYN), so any design
+  can be tested under SYN before its config.mk officially switches.
+- `<name>_yosys_*` pins the yosys engine, so the baseline stays
+  reproducible after a design switches (asap7/gcd today).
+
+Designs with a `rules-base.json` get two QoR tiers per variant:
+
+- `<name>_<engine>_synth_test` — the fast tier: synthesis only,
+  checked against the `synth__`/`constraints__` subset of the rules.
+  Minutes. QoR-only — without LEC it cannot prove correctness, so use
+  it to iterate, not to conclude.
+- `<name>_<engine>_test` — the full flow against all rules: the real,
+  dashboard-equivalent gate. It reuses the fast tier's synthesis
+  action, so escalating from fast to full never repeats synthesis.
+
+## The loop
+
+Build/test whatever is relevant to what you are changing — one design,
+one stage, one tier, or a whole suite:
+
+```sh
+# fastest signal for one design
+bazelisk test //flow/designs/asap7/jpeg:jpeg_encoder_syn_synth_test
+
+# the real gate for that design
+bazelisk test //flow/designs/asap7/jpeg:jpeg_encoder_syn_test
+
+# a single stage, for triage
+bazelisk build //flow/designs/asap7/jpeg:jpeg_encoder_syn_route
+
+# yosys A/B on an officially-SYN design
+bazelisk test //flow/designs/asap7/gcd:gcd_yosys_test
+
+# suite-wide status (asap7): fast tier, then the full flows
+bazelisk test --keep_going //flow/designs/asap7:syn_synth_test
+bazelisk test --keep_going //flow/designs/asap7:syn_test
+```
+
+Then display what you have — deliberately a separate concern from
+deciding what to test or build:
+
+```sh
+bazelisk run //flow/designs:syn-dashboard
+```
+
+It prints a GitHub-markdown TL;DR table of every forced-engine test
+result already in `bazel-testlogs` — pass/fail per tier and engine,
+failing rule fields, result age — without building anything. Paste it
+into a PR or issue as-is. Because the display reads whatever exists,
+the loop stays fast: iterate on the one target you care about, and the
+dashboard aggregates across everything you (or earlier sessions) have
+built so far.
+
+The suites include designs that fail OpenROAD-SYN today on purpose —
+they are a status signal, not a green-only gate. A build failure (e.g.
+synthesis dies) records no test result at all, so run the suites with
+`--keep_going` and check their output for `FAILED TO BUILD`; designs
+too slow for the suites are listed in `SYN_SLOW_DESIGNS`
+(`asap7/BUILD`) and reproduced individually.

flow/designs/asap7/BUILD

@@ -0,0 +1,111 @@
+# OpenROAD-SYN test suites for asap7 — the local counterpart of the
+# Jenkins QoR signal (https://dashboard.precisioninno.com/).
+#
+# Every design gets manual forced-engine flow variants from the
+# config.mk DSL (see flow/designs/design.bzl): <name>_syn_* pins
+# SYNTH_USE_SYN=1 and <name>_yosys_* pins the yosys engine, regardless
+# of the config.mk default. Designs with a rules-base.json get two QoR
+# gates per variant, both checking the design's regular rules file with
+# the regular checkMetadata.py — congruent with what Jenkins measures:
+#
+#   bazelisk test --keep_going //flow/designs/asap7:syn_synth_test
+#     Fast tier: synthesis only, checks the synth__/constraints__ subset
+#     of rules-base.json. Minutes-scale; QoR-only (no LEC), so it cannot
+#     prove correctness — use it to iterate, not to conclude.
+#
+#   bazelisk test --keep_going //flow/designs/asap7:syn_test
+#     Full flow, all rules-base.json gates. The real signal; shares its
+#     synthesis action with the fast tier.
+#
+# Sharpening the loop on one design or stage:
+#
+#   bazelisk test //flow/designs/asap7/jpeg:jpeg_encoder_syn_synth_test
+#   bazelisk test //flow/designs/asap7/jpeg:jpeg_encoder_syn_test
+#   bazelisk build //flow/designs/asap7/jpeg:jpeg_encoder_syn_route
+#   bazelisk test //flow/designs/asap7/gcd:gcd_yosys_test   # yosys A/B
+#
+# Both suites deliberately include designs that fail OpenROAD-SYN today
+# — they are a status signal, not a green-only gate. A design can fail
+# by not synthesizing at all, or by missing rules-base.json gates that
+# its regular flow meets.
+
+# Design directory -> DESIGN_NAME (bazel target prefix), ordered
+# fastest-first so --keep_going runs surface quick failures early.
+# Known failures / quirks are listed last. "minimal" is excluded: it
+# has no VERILOG_FILES and thus no generated design targets.
+#
+# Known OpenROAD-SYN failures today:
+# - mock-alu: SDC references yosys-style register names (*io_out_REG*)
+#   absent from the SYN netlist.
+# - riscv32i: slang rejects procedural assignment to nets (dmem.v).
+# - mock-cpu: slang rejects endianness-mismatched part-select
+#   (wptr_full.v).
+# - aes-block: hierarchical BLOCKS; synthesizes, but SYN area blows past
+#   the yosys-derived rules-base.json gate.
+SYN_TEST_DESIGNS = {
+    "gcd": "gcd",  # smoketest, fast
+    "uart": "uart",
+    "mock-alu": "MockAlu",
+    "riscv32i": "riscv_top",
+    "aes": "aes_cipher_top",
+    "aes_lvt": "aes_cipher_top",
+    "aes-mbff": "aes_cipher_top",
+    "jpeg": "jpeg_encoder",
+    "jpeg_lvt": "jpeg_encoder",
+    "ethmac": "ethmac",
+    "ethmac_lvt": "ethmac",
+    "mock-cpu": "mock_cpu",
+    "gcd-ccs": "gcd",  # CCS liberty
+    "aes-block": "aes_cipher_top",  # hierarchical BLOCKS
+}
+
+# Designs whose OpenROAD-SYN run is too slow (or does not terminate) for
+# the suites. Their generated per-design targets stay tags=["manual"]
+# like all forced-engine targets, but they are deliberately left out of
+# the suites so those remain runnable; reproduce individually, e.g.
+#   bazelisk build //flow/designs/asap7/ibex:ibex_core_syn_synth
+#
+# - ibex: sv_elaborate/synthesize/repair_design complete in seconds, but
+#   report_metrics ("Report metrics stage 1, synth...") spins in STA for
+#   hours (observed >2h on 48 threads with no progress).
+# - cva6: synthesize itself takes ~73s, but repair_design -pre_placement
+#   was still running after 10+ minutes when the status sweep was cut
+#   off; not yet characterized to completion.
+# - swerv_wrapper: largest design in the suite; not yet run to
+#   completion under OpenROAD-SYN in this environment.
+# - riscv32i-mock-sram: hierarchical BLOCKS (fakeram) — the syn variant
+#   first needs the block's full PnR flow for its abstract; not yet run
+#   to completion.
+SYN_SLOW_DESIGNS = {
+    "cva6": "cva6",
+    "ibex": "ibex_core",
+    "riscv32i-mock-sram": "riscv_top",
+    "swerv_wrapper": "swerv_wrapper",
+}
+
+# Explicitly listed manual tests DO run through a test_suite (the manual
+# tag only exempts them from wildcard expansion); the suites themselves
+# are manual so `bazelisk test //flow/...` does not pull them in either.
+test_suite(
+    name = "syn_test",
+    tags = ["manual"],
+    tests = [
+        "//flow/designs/asap7/{pkg}:{name}_syn_test".format(
+            name = name,
+            pkg = pkg,
+        )
+        for pkg, name in SYN_TEST_DESIGNS.items()
+    ],
+)
+
+test_suite(
+    name = "syn_synth_test",
+    tags = ["manual"],
+    tests = [
+        "//flow/designs/asap7/{pkg}:{name}_syn_synth_test".format(
+            name = name,
+            pkg = pkg,
+        )
+        for pkg, name in SYN_TEST_DESIGNS.items()
+    ],
+)

flow/designs/asap7/gcd/BUILD

@@ -3,13 +3,13 @@ load("@orfs_designs//:designs.bzl", "DESIGNS")
 load("@rules_shell//shell:sh_test.bzl", "sh_test")
 load("//flow/designs:design.bzl", "design")
 
-# SYNTH_USE_SYN is make-only for now: bazel-orfs's synth stage always
-# runs the yosys flow and neither stages the Verilog sources nor sets
-# VERILOG_FILES for the OpenROAD synthesis step, so the built-in
-# synthesizer opt-in must not reach the bazel arguments.
+# gcd's regular flow follows config.mk onto OpenROAD-SYN
+# (SYNTH_USE_SYN=1). The single-process comparison below deliberately
+# exercises the yosys netlist path (single_flow.tcl seeds from
+# 1_2_yosys.v), so it consumes the forced-yosys gcd_yosys_* variant the
+# config.mk DSL emits for every design (see flow/designs/design.bzl).
 design(
     config = "config.mk",
-    local_arguments = ["SYNTH_USE_SYN"],
 )
 
 # Stage-boundary .odb/.sdc stems shared by the normal per-stage flow
@@ -56,7 +56,7 @@ SINGLE_FLOW_EXTRA_OUTS = [
 # this package.
 orfs_run(
     name = "gcd_single_flow",
-    src = ":gcd_synth",
+    src = ":gcd_yosys_synth",
     outs = [
         "results/asap7/gcd/single/{}.{}".format(stem, ext)
         for stem in SINGLE_FLOW_STAGES.values()
@@ -79,11 +79,13 @@ orfs_run(
     variant = "single",
 )
 
-# Normal-flow stage outputs, one filegroup per compared file.
+# Normal-flow stage outputs, one filegroup per compared file. Taken from
+# the yosys-engine variant above so both sides of the byte-compare start
+# from the same yosys netlist.
 [
     filegroup(
         name = "gcd_base_{}_{}".format(stage, ext),
-        srcs = [":gcd_" + stage],
+        srcs = [":gcd_yosys_" + stage],
         output_group = "{}.{}".format(stem, ext),
     )
     for stage, stem in SINGLE_FLOW_STAGES.items()

flow/designs/design.bzl

@@ -1,5 +1,6 @@
 """BUILD boilerplate for flow/designs/."""
 
+load("@bazel-orfs//:openroad.bzl", "orfs_flow")
 load("@orfs_designs//:designs.bzl", "orfs_design")
 
 # Per filegroup target: extensions included in the filegroup.
@@ -51,6 +52,65 @@ def _export_design_files():
         visibility = ["//visibility:private"],
     )
 
+def _engine_variant_targets(
+        name,
+        platform,
+        verilog_files,
+        arguments,
+        user_arguments,
+        sources,
+        user_sources,
+        macros,
+        stage_data,
+        tags):  # buildifier: disable=unused-variable
+    """Forced-engine flow variants for one design (orfs_design extra hook).
+
+    Emits, always tagged "manual" (even for CI designs, so only explicit
+    invocations such as //flow/designs/asap7:syn_test run them), two full
+    flows that pin the synthesis engine regardless of the config.mk
+    default:
+
+    - variant "syn": SYNTH_USE_SYN=1 (OpenROAD-SYN) — lets any design be
+      tested under OpenROAD-SYN before its config.mk officially switches.
+    - variant "yosys": SYNTH_USE_SYN=0 — keeps the yosys baseline
+      reproducible after a design officially switches (e.g. asap7/gcd).
+
+    When the design has a rules-base.json each variant gets the same QoR
+    gates as the regular flow, congruent with the Jenkins/dashboard
+    signal because they check the same rules file with the same script:
+
+    - <name>_<engine>_test: the full flow gated on all of rules-base.json
+      (the real, dashboard-equivalent gate).
+    - <name>_<engine>_synth_test: the fast tier — synthesis plus a check
+      of only the synth__/constraints__ subset of rules-base.json.
+      QoR-only (no LEC), but a minutes-scale iteration signal that shares
+      its synth action with the full-flow variant.
+
+    The generated <name>_<engine>_update targets must NOT be run: they
+    would overwrite the design's rules-base.json with the forced engine's
+    numbers.
+    """
+    for engine, use_syn in [("syn", "1"), ("yosys", "0")]:
+        orfs_flow(
+            name = name,
+            verilog_files = verilog_files,
+            pdk = "//flow:" + platform,
+            arguments = arguments | {"SYNTH_USE_SYN": use_syn},
+            user_arguments = user_arguments,
+            # sources carries the auto-detected rules-base.json as
+            # RULES_JSON, which gates the variant's tests.
+            sources = sources,
+            user_sources = user_sources,
+            macros = macros,
+            stage_data = stage_data,
+            variant = engine,
+            tags = ["manual"],
+            test_kwargs = {"tags": ["manual"]},
+            # Sibling packages consume stage outputs (e.g. asap7/gcd's
+            # single-process comparison seeds from the yosys netlist).
+            visibility = ["//visibility:public"],
+        )
+
 def design(config = "config.mk", user_arguments = [], user_sources = [], local_arguments = []):
     """Standard BUILD body for flow/designs/<platform>/<design>/.
 
@@ -75,6 +135,7 @@ def design(config = "config.mk", user_arguments = [], user_sources = [], local_a
         user_sources = user_sources,
         local_arguments = local_arguments,
         blender = True,
+        extra = _engine_variant_targets,
     )
 
 def files(group, extra_srcs = None):

flow/scripts/synth_syn.tcl

@@ -75,3 +75,12 @@ orfs_write_db $::env(RESULTS_DIR)/1_synth.odb
 # out by OpenSTA that has no dependencies. Sole writer of
 # 1_synth.sdc.
 orfs_write_sdc $::env(RESULTS_DIR)/1_synth.sdc
+
+# Gate-level netlist for LEC (write_lec_verilog strips physical-only
+# masters, matching the CTS-stage LEC convention) and any other netlist
+# consumer. The Bazel synthesis action declares this file as an output,
+# so it must be written unconditionally. write_lec_verilog takes a bare
+# filename and prefixes $::env(RESULTS_DIR) itself, like the cts.tcl
+# call sites.
+source $::env(SCRIPTS_DIR)/lec_check.tcl
+write_lec_verilog 1_synth.v

flow/util/BUILD

@@ -1,4 +1,4 @@
-load("@rules_python//python:defs.bzl", "py_library", "py_test")
+load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
 load("@rules_python//python:pip.bzl", "compile_pip_requirements")
 
 exports_files(["open_plots.sh"])
@@ -33,6 +33,14 @@ filegroup(
     visibility = ["//visibility:public"],
 )
 
+# TL;DR markdown dashboard over locally built OpenROAD-SYN QoR tests;
+# run via the //flow/designs:syn-dashboard alias.
+py_binary(
+    name = "synDashboard",
+    srcs = ["synDashboard.py"],
+    visibility = ["//flow/designs:__pkg__"],
+)
+
 py_library(
     name = "genMetrics_lib",
     srcs = ["genMetrics.py"],

flow/util/checkMetadata.py

@@ -33,6 +33,14 @@
 )
 parser.add_argument("--metadata", "-m", required=True, help="The metadata file")
 parser.add_argument("--rules", "-r", required=True, nargs="+", help="The rules file")
+parser.add_argument(
+    "--only-prefix",
+    nargs="+",
+    default=None,
+    help="Check only rules whose field starts with one of these prefixes. "
+    "Lets a partial run (e.g. synthesis-only) be gated by a full-flow "
+    "rules file without tripping the missing-field error.",
+)
 args = parser.parse_args()
 
 with open(args.metadata) as metadataFile:
@@ -46,6 +54,13 @@
     else:
         print(f"[WARN] File {filePath} not found")
 
+if args.only_prefix:
+    rules = {
+        field: rule
+        for field, rule in rules.items()
+        if any(field.startswith(prefix) for prefix in args.only_prefix)
+    }
+
 if len(rules) == 0:
     print("No rules")
     sys.exit(1)