[Release 0.70.0] ~85% NumPy 2.x API coverage: 100+ new np.* APIs, new nuget packages (OpenBLAS & pythonnet), 3 living dashboards

.claude/skills/benchmark/SKILL.md

@@ -2,7 +2,8 @@
 name: benchmark
 description: >-
   NumSharp's NumPy-vs-NumSharp performance harness — the op/dtype/N matrix (C# BenchmarkDotNet vs a
-  warm NumPy process) plus five appended subsystems (nditer, layout, operand, cast, fusion), all in
+  warm NumPy process), unified Managed/OpenBLAS backend profiles, and five appended subsystems
+  (nditer, layout, operand, cast, fusion), all in
   the NPY/NS convention. Use this whenever you add a benchmark for an np.* op, wire a C# benchmark to
   its NumPy twin, run the official suite or a subset, read/interpret the ratio matrix or history
   snapshots, add a whole subsystem, or debug a suspicious measurement (the Debug-taint 2x pitfall,
@@ -21,7 +22,7 @@ is the distilled map + the actionable playbooks. Read `benchmark/CLAUDE.md` when
 
 > **ratio = NumPy_ms / NumSharp_ms.** `>1` = NumSharp **faster**, `<1` = slower, `=1` = parity. **Higher is better.**
 
-Used everywhere — matrices, geomeans, commit messages, every `*_sheet.py`. Icons: ✅ `≥1.0` · 🟡 `≥0.5` · 🟠 `≥0.2`
+Used everywhere — matrices, geomeans, commit messages, every `*_sheet.py`. Dashboard/report bands: ✅ `≥1.05` · 🟡 `≥0.5` · 🟠 `≥0.2`
 · 🔴 `<0.2`. (The legacy `run-benchmarks.ps1` prints the INVERSE NS/NPY — prefer NPY/NS for anything new.)
 
 ## THE pitfall: Debug taints timings ~2×
@@ -38,7 +39,7 @@ script assembly, not Core. The BenchmarkDotNet projects are exempt (they mandate
 |------|-------|------|
 | C# | `benchmark/NumSharp.Benchmark.CSharp/Benchmarks/<Category>/*.cs` | BenchmarkDotNet classes; `[Benchmark(Description="np.foo(a)")]` methods. |
 | NumPy | `benchmark/NumSharp.Benchmark.Python/numpy_benchmark.py` | `run_<suite>_benchmarks(...)` emitting `BenchmarkResult` rows. |
-| Merge | `benchmark/scripts/merge-results.py` | Joins on `(normalize_op_name(name), dtype, N)`. |
+| Merge | `benchmark/scripts/{merge-results,merge-backend-profiles}.py` | Joins language timings, then backend profiles on the exact cell. |
 | Orchestrator | `benchmark/run_benchmark.py` | Builds C#, runs each suite (BDN) + warm NumPy across 1K/100K/10M, merges, snapshots. |
 
 The join is by **normalized op name**: `normalize_op_name` strips the dtype tag, `[annotations]`, and
@@ -47,12 +48,17 @@ names to normalize identically or the row shows as "C# not run" / "NumPy only".
 
 ## The matrix + subsystems
 
-- **Op matrix** — 14 comparison suites, each a C# namespace filter in `run_benchmark.py`'s `SUITES` map
+- **Op matrix** — 18 comparison suites, each a C# namespace filter in `run_benchmark.py`'s `SUITES` map
   (`arithmetic, unary, reduction, broadcast, creation, manipulation, slicing, comparison, bitwise, logic,
-  statistics, sorting, linalg, selection`). Swept over 1K/100K/10M × the 15 dtypes.
+  statistics, sorting, linalg, selection, fft, random, ndarray, api`). Swept over the size/dtype tiers
+  each API can exercise safely; the classic kernels retain the 1K/100K/10M sweep.
+- **Backend profiles** — `benchmark/backends/backend_profiles.py` runs every backend-sensitive API
+  under Managed C# and OpenBLAS with one schema; MissingBackendException and NotSupportedException
+  are availability outcomes. Separate profile JSON files are merged into one effective dataset.
 - **Five appended subsystems** (own result models, appended not merged): `nditer` (iterator machinery),
   `layout` (op × 8 memory layouts × dtype), `operand` (1-D/scalar/mixed/broadcast), `cast` (astype 15×15 × layout),
-  `fusion` (`np.evaluate`). Each is a `*_bench.{cs,py}` pair + a `*_sheet.py` renderer.
+  and `fusion` (`np.evaluate`). Each is a
+  `*_bench.{cs,py}` pair + a `*_sheet.py` renderer.
 
 ## Playbook — add a benchmark for a new op
 
@@ -71,8 +77,8 @@ The most common task. Full worked example in **`references/add-benchmark.md`**.
 
 ## Other tasks → where to go
 
-- **Run the suite (official / subset), interpret the report, snapshots** → `references/run-and-report.md`.
-- **Add or edit a matrix subsystem (nditer/layout/operand/cast/fusion)** → `references/subsystems.md`.
+- **Run the suite (official / subset), interpret the report, the reports/UI surfaces + snapshots** → `references/run-and-report.md`. (The human-facing UI is the DocFX page `docs/website-src/docs/benchmarks-dashboard.md`; its Function Explorer data is generated, while narrative cards are curated.)
+- **Add or edit a matrix subsystem or backend profile case** → `references/subsystems.md`.
 - **Everything else (all suites, config internals, troubleshooting, type map)** → `benchmark/CLAUDE.md`.
 
 ## Gotchas
@@ -91,4 +97,4 @@ The most common task. Full worked example in **`references/add-benchmark.md`**.
 
 - `references/add-benchmark.md` — the detailed add-a-benchmark playbook (C# + NumPy twin + join-key rules + smoke).
 - `references/run-and-report.md` — running the official run / subsets, the report + history snapshots, InProcessEmit.
-- `references/subsystems.md` — the five subsystems and how to add one (`*_bench.{cs,py}` + `*_sheet.py`).
+- `references/subsystems.md` — backend profiles plus the five appended subsystems.

.claude/skills/benchmark/references/add-benchmark.md

@@ -121,7 +121,7 @@ Only when your ops form a new category with no existing home:
 4. (legacy) optionally add the filter to `run-benchmarks.ps1`'s `ValidateSet` and Program.cs menu.
 
 Then the new suite runs in `python run_benchmark.py --suites <newsuite>` (and in the full run), joined and reported
-exactly like the built-in 14. Keep the C# `[Benchmark(Description)]` labels normalizing onto the NumPy `.name`s.
+exactly like the built-in 18. Keep the C# `[Benchmark(Description)]` labels normalizing onto the NumPy `.name`s.
 
 ## Choosing base class & size (quick reference)
 

.claude/skills/benchmark/references/run-and-report.md

@@ -3,20 +3,22 @@
 ## Official run — `benchmark/run_benchmark.py` (the entry point)
 
 Builds the C# suite, runs each suite through BenchmarkDotNet (per-class JSON, resumable), sweeps warm NumPy across
-1K/100K/10M, merges on `(op, dtype, N)`, appends the five subsystems, archives raw scratch to
+the applicable size tiers, runs Managed/OpenBLAS profiles, merges on `(op, dtype, N, scenario)`,
+appends the five complementary subsystems, and archives raw scratch to
 `results/<ts>/` (gitignored), and writes the committable `history/<date>_<sha>/` snapshot.
 
 ```bash
-python run_benchmark.py                       # full official run (all 14 suites + 5 subsystems)
+python run_benchmark.py                       # full official run (18 suites + backend profiles + subsystems)
 python run_benchmark.py --suites manipulation unary   # subset of the op matrix
 python run_benchmark.py --skip-build          # reuse existing Release build
 python run_benchmark.py --skip-csharp         # NumPy only
 python run_benchmark.py --quick               # dev: fewer NumPy iterations
 python run_benchmark.py --no-history          # don't write the history snapshot
 # subsystem opt-outs: --skip-nditer --skip-layout --skip-operand --skip-cast --skip-fusion
+# backend profile opt-out: --skip-openblas
 ```
 
-**Cost:** the full matrix is long (µs–ms array ops × 15 dtypes × 3 sizes × 14 suites + subsystems). For iterating
+**Cost:** the full matrix is long (µs–ms array ops × applicable dtypes/sizes × 18 suites + subsystems). For iterating
 on one op, `--suites <that suite>` or the smoke path (`--list flat` + `numpy_benchmark.py --suite <s> --quick`) is
 usually the right scope. A full measured run + committed snapshot is the post-release `.github/workflows/
 benchmark.yml` ritual, not something to kick off casually.
@@ -37,19 +39,37 @@ The nditer subsystem reports a section that crashes all retries (the known inter
 
 ## Reading the report
 
-- **Convention is NPY/NS** (NumPy_ms / NumSharp_ms, `>1` = NumSharp faster). Icons ✅ `≥1.0` 🟡 `≥0.5` 🟠 `≥0.2`
-  🔴 `<0.2`.
-- The report has a **per-size geomean summary** + the full **per-(op, dtype, N) ratio matrix**, then the five
-  appended subsystem sections.
+- **Convention is NPY/NS** (NumPy_ms / NumSharp_ms, `>1` = NumSharp faster). Published bands: ✅ `≥1.05×` 🟡 `≥0.5×`
+  🟠 `≥0.2×` 🔴 `<0.2×` · **▫ negligible** (sub-µs either side or >20× — excluded from geomeans & Best/Worst)
+  · **⚪** (C# side unjoined). The `%NumPy🕐` column = NumSharp_ms / NumPy_ms × 100 = share of NumPy's time
+  NumSharp uses (<100% = faster).
+- **Credibility gating** (`merge-results.py` `classify()`): only rows where **both sides did ≥1µs of work AND the
+  speedup is within 20×** count toward the geomeans and rankings. Sub-µs call-overhead rows, view returns, lazy
+  allocs and dead-code-eliminated kernels are `▫ negligible` — kept in the per-suite tables, never showcased.
+- The report has the full **per-(op, dtype, N, scenario) matrix**, both profile results, one
+  fastest-valid effective value, then the five appended subsystem sections.
 - A row missing a C# or NumPy value ("C# not run" / "NumPy only") almost always means the two names didn't
   **normalize to the same join key** — check the C# `[Benchmark(Description)]` vs the NumPy `.name`.
 
+## Reports & UI surfaces (canonical → human-facing)
+
+They drift — know which is which:
+- **`benchmark/benchmark-report.md`** — the canonical backend-aware report. Tracked; refreshed by CI. Start here.
+- **`benchmark-report.{managed,openblas}.json`** — separate profiles using the same schema; the unsuffixed
+  JSON contains both profiles plus the effective selection.
+- **`benchmark/history/latest/*`** — the committable snapshot the docs/CI reference.
+- **`benchmark/benchmark-dashboard.md`** — a dense ASCII-bar sheet from `scripts/render_dashboard.py`. Gitignored,
+  **NOT** wired into `run_benchmark.py` or CI — run it by hand to seed the DocFX dashboard's numbers.
+- **`docs/website-src/docs/benchmarks-dashboard.md`** — the **real UI**, promoted from the backend POC. It reads
+  only the canonical combined JSON and computes effective rollups/backend drill-downs from measured rows.
+- **`benchmark/README.md`** is a static orientation guide, **not** the report — CI never refreshes it.
+
 ## History snapshots — what we commit
 
 | Path | Tracked? | Contents |
 |------|----------|----------|
 | `benchmark/results/<ts>/` | ❌ gitignored | raw per-run scratch (per-suite NumPy JSON, BDN per-class reports, merged json/csv). |
-| `benchmark/history/<date>_<sha>/` | ✅ tracked | the snapshot: `MANIFEST.md` + `benchmark-report.{md,json,csv}` + `numpy-results.json` + every subsystem `*_results.{md,tsv}` + `cards/`. |
+| `benchmark/history/<date>_<sha>/` | ✅ tracked | the snapshot: MANIFEST + combined/separate profile JSON + report/csv + NumPy input + subsystem results + cards. |
 | `benchmark/history/latest` | ✅ tracked symlink | → the newest snapshot. Stable path for docs/CI. |
 
 `benchmark/scripts/snapshot_history.py` assembles it (called by `run_benchmark.py`; `--commit` to also git-commit).

.claude/skills/benchmark/references/subsystems.md

@@ -1,8 +1,7 @@
-# The five matrix subsystems (and how to add one)
+# Backend profiles and the five complementary subsystems
 
-Beyond the op/dtype/N matrix, `run_benchmark.py` appends five subsystems that fill axes the op matrix can't
-express. Each has its OWN result model and is **appended, not merged** into the op matrix. They live in
-`benchmark/{nditer,layout,operand,cast,fusion}/`.
+`run_benchmark.py` first merges targeted Managed/OpenBLAS profile rows into the op matrix, then appends
+five subsystems whose result models cannot fit that matrix. Backend exceptions are availability states.
 
 | Subsystem | Dir | What it adds the op matrix lacks | Result model |
 |-----------|-----|----------------------------------|--------------|
@@ -11,15 +10,18 @@ express. Each has its OWN result model and is **appended, not merged** into the
 | **operand** | `benchmark/operand/` | 1-D (contig/strided/reversed), scalar operand, mixed operand layouts (C+F, C+T), binary broadcast | case × dtype |
 | **cast** | `benchmark/cast/` | full `astype` src→dst × 8 layouts at 1M — no op-matrix coverage at all | 15×15 per-layout matrices |
 | **fusion** | `benchmark/fusion/` | `np.evaluate` fused vs unfused np.* chains (+ NumPy context) | fixed-expression report |
+| **backend profiles** | `benchmark/backends/` → `benchmark/openblas/` | all 39 backend-sensitive APIs under Managed C# and OpenBLAS; catches MissingBackendException and NotSupportedException | op × dtype × N × scenario, merged |
 
-## The shared shape (every subsystem is the same three files)
+## The shared shape (the five appended subsystems)
 
 1. **`<name>_bench.cs`** — a NumSharp file-based app fed on stdin via `dotnet run -c Release -`. Emits rows with the
    subsystem's keys. Its author-absolute `#:project` path is rewritten to the running checkout by the driver.
 2. **`<name>_bench.py`** — the NumPy twin emitting IDENTICAL keys.
 3. **`<name>_sheet.py`** — merges the two and renders `<name>_results.md` (+ `.tsv`), in the **NPY/NS** convention.
 
-The build/run/parse plumbing is shared in `benchmark/scripts/bench_common.py`. `run_benchmark.py` calls each
+The build/run/parse plumbing is shared in `benchmark/scripts/bench_common.py`. The backend driver uses it too,
+but writes same-schema `openblas_results.{managed,openblas}.json` consumed by
+`merge-backend-profiles.py`. `run_benchmark.py` calls each
 `*_sheet.py` and appends its `*_results.md` as a report section; `--skip-<name>` opts out.
 
 ## Adding a new op to an EXISTING subsystem