From cd9f3f60be423b4da9f1984e17062a6c1a774c08 Mon Sep 17 00:00:00 2001 From: Michael Feth Date: Fri, 21 Aug 2026 16:39:28 -0400 Subject: [PATCH] Thread placement on one CCD, and the default that is two per core MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `--cpus` was measured on a two-CCD Ryzen 9 9900X, where crossing the die costs 25% and reproduces to 0.5%. This adds the case one step in: a Ryzen 7 3700X, Zen 2, one CCD holding two 4-core CCXs with separate 16 MB L3. Kimi-Linear, cache saturated at --budget 18G so every arm reports identical hit rate and identical bytes, WASTE_XPAR=1, three repeats per arm. Crossing a CCX does not reproduce the die result, and the honest reading is "below what this host can resolve" rather than "cheaper". The grid carried two arms that were the same configuration as a control; they came back 3.38 and 3.72, so anything under ~10-12% is noise here. A second run of the identical grid ordered the two CCXs the other way round. What the data bounds is the size: not 25%, and not measurable at this precision. Left as a bound, because a number that flips sign between runs is not a finding. Thread count moves much more on the same machine, and the default is the wrong end of it: 16 threads (the default here) 2.31 tok/s 8 threads 3.38-3.72 6 threads 3.76 `waste_cpu_count()` counts logical CPUs — GetActiveProcessorCount on Windows, _SC_NPROCESSORS_ONLN on Linux — so `--threads 0` on an 8-core part with SMT is 16, two per core, and ~1.6x slower than 6-8. The apply is a dependent load -> address -> load chain: two siblings on one core split its L1 and load/store ports and add no memory-level parallelism. That is §47's "saturates at six threads" reaching x86 from the other side. 6 and 8 are one plateau, inside each other's noise; only the default is off it. The CLI help said `0 (default) is one per core`. It is one per logical CPU, which is the whole reason the default lands where it does, so it now says that. Also recorded, as an observation rather than a conclusion: the first repeat of every arm is the slowest. #37 reports the same shape on a Strix Halo and reads it as warming. Two machines, same pattern -- hence medians over three repeats here rather than a best-of. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01B5eVMiauR4Lkt8Dhc67MNb --- cli/main.c | 3 ++- docs/ENGINE.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 1 deletion(-) diff --git a/cli/main.c b/cli/main.c index ae2484e..f5822cc 100644 --- a/cli/main.c +++ b/cli/main.c @@ -1240,7 +1240,8 @@ int main(int argc, char **argv) " info and bench\n" " --learn records which experts the run used, so the next open\n" " starts with a warm cache instead of an empty one\n" - " --threads sets the compute pool; 0 (default) is one per core\n" + " --threads sets the compute pool; 0 (default) is one per\n" + " logical CPU, so two per core where SMT is on\n" " --cpus restricts that pool to a cpu list — \"0-5\", \"0-2,6-8\" —\n" " and then --threads 0 means one per CPU listed. Worth it where\n" " the cores differ: on a two-die Ryzen, six threads on one die\n" diff --git a/docs/ENGINE.md b/docs/ENGINE.md index 286aec8..3ec84e7 100644 --- a/docs/ENGINE.md +++ b/docs/ENGINE.md @@ -383,6 +383,55 @@ alone reproduces the first row to 0.5%, and the penalty survives `WASTE_XPAR=1` unchanged (−24.9% against −25.0%), so it is a property of where the threads run rather than of how the work is cut. +**How far that generalises: not to a CCX.** Third-party measurement on a +Ryzen 7 3700X — Zen 2, **one** CCD holding two 4-core CCXs with separate +16 MB L3 — Kimi-Linear, cache saturated at `--budget 18G` so every arm +reports the same hit rate and the same bytes, `WASTE_XPAR=1`, three repeats +per arm, medians: + +| 8 threads on | median tok/s | +|---|---| +| wherever the OS puts them | 3.72 | +| `0-7`, one CCX | 3.10 | +| `8-15`, the other CCX | 2.42 | +| `0-3,8-11`, split 4+4 | 2.81 | + +Read that as **no measurable effect, not as a penalty.** The same run +included two arms that were the *same* configuration as a control, and they +came back 3.38 and 3.72 — so anything under about 10–12% is this machine's +noise, and a second run of the identical grid put the two CCXs in the +opposite order. What the numbers bound is the size: whatever crossing a CCX +costs on a single-die part, it is not the 25% that crossing a *die* costs +above, and it is below what this host can resolve. The option is worth +reaching for on a multi-CCD machine; on one CCD, leaving placement to the OS +is not leaving anything on the table. + +**Thread count moves far more than placement does, on the same machine.** +Same container, same budget, same control: + +| threads | median tok/s | +|---|---| +| 16 — *the default here* | 2.31 | +| 8 | 3.38–3.72 | +| 6 | 3.76 | + +`--threads 0` is one per CPU the pool may use, and `waste_cpu_count()` counts +**logical** CPUs — `GetActiveProcessorCount(ALL_PROCESSOR_GROUPS)` on Windows, +`sysconf(_SC_NPROCESSORS_ONLN)` on Linux. On an 8-core part with SMT on that +is 16, two per core, and it is **~1.6x slower than 6–8**. The apply is a +dependent `load → address → load` chain, so two siblings on one core split +its L1 and load/store ports without adding memory-level parallelism — §25's +"the apply saturates at six threads" arriving on x86 from the other side. +The plateau at 6 and 8 is one plateau: those two are inside the noise of each +other, and only the default is clearly off it. + +One observation offered rather than concluded: the **first repeat of each arm +is consistently the slowest** — 1.81 before 2.71/2.31 at 16 threads, 2.60 +before 3.38/3.42 at 8. [issue #37](https://github.com/sqliteai/warp/issues/37) +reports the same shape on a Strix Halo and reads it as warming rather than as +an outlier. Two machines, same pattern, so medians over three repeats rather +than a best-of. + `docs/LEARNED.md` §47 is the same mechanism on a different axis — a pool spanning P-cores and E-cores, where the slow participant is slow by speed rather than by distance. Both come back to the same structure: