From adab190b3e0ce46d5e9c64249a3d4cfb6ee8888f Mon Sep 17 00:00:00 2001 From: Michael Feth Date: Fri, 21 Aug 2026 14:19:42 -0400 Subject: [PATCH] diskbench narrowed its offset to off_t, so 8 GB of file was 2 GiB of reads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit waste_pread takes an int64_t offset, and on Windows splits it into ReadFile's Offset/OffsetHigh pair, precisely so a read past 2 GiB works. The random-read loop handed it an off_t instead. Under LLP64 — MSYS2 UCRT64, the documented Windows build — off_t is a 32-bit long while g_file and g_rec are size_t, so `(off_t)(seed % nrec) * g_rec` truncates. The default file size is 8 GB, so this was the default path, not a corner. Two failure modes, and the quiet one is worse. Offsets that wrapped negative were refused by waste_pread's `off < 0` guard, which returns -1 without touching errno; the loop prints "short read -1: No error" and breaks, so that thread stops early and the bytes it never read still divide into its elapsed time. Offsets that wrapped positive did not fail at all — they read the wrong place, successfully. A run kept its whole working set inside the first 2 GiB while reporting the size it was asked for, and a 2 GiB hot region sits inside a consumer SSD's SLC cache, which is exactly the flattery this tool exists to avoid. docs/GATES.md Gate H sized the storage premise on this program. Measured on Windows 11 / Ryzen 7 3700X / MSYS2 UCRT64 / gcc 16.2.0, WD_BLACK SN7100 4 TB. Same path, same parameters, the two runs back to back: diskbench PATH 8 12 4 before after short reads 7 x "-1: No error" none rand 1 thr 0.36 GB/s 1.48 GB/s rand 2 thr 0.11 GB/s 2.23 GB/s rand 4 thr 0.06 GB/s 2.25 GB/s seq write / seq read 2.25 / 2.17 2.21 / 2.13 The sequential columns barely move, which is what makes the pair comparable — only the random path changed. The before column is not a slow disk and not even a stable wrong answer: it *inverts* with thread count, because each thread breaks out of its loop at whatever offset first truncates, and more threads means that happens sooner. An earlier run of the same unmodified binary gave 0.34 / 0.80 / 1.02; the garbage is not reproducible, which is its own argument. At 2 GB, where nothing truncates, it reports a clean monotonic 1.58 / 2.00 / 2.33. Absolute throughput here is depressed by an unrelated model conversion running on the same drive. That load is common to both columns and the sequential rows show it; the numbers are a before/after pair, not a device rating. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01B5eVMiauR4Lkt8Dhc67MNb --- tools/diskbench.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/tools/diskbench.c b/tools/diskbench.c index 3b80b53..0e02a37 100644 --- a/tools/diskbench.c +++ b/tools/diskbench.c @@ -247,7 +247,20 @@ static void *rand_reader(void *p) { double got = 0; for (int i = 0; i < g_reps; i++) { seed = seed * 1103515245u + 12345u; - off_t off = (off_t)(seed % nrec) * g_rec; + /* Not off_t: under LLP64 — MSYS2 UCRT64, the documented Windows + * build — off_t is a 32-bit long, while g_file and g_rec are size_t + * and waste_pread takes an int64_t precisely so that an offset past + * 2 GiB survives the trip to ReadFile's Offset/OffsetHigh pair. + * Narrowing through off_t here threw that away at the last step: the + * offsets that wrapped negative were refused by waste_pread's + * `off < 0` guard, which returns -1 without touching errno and prints + * as "short read -1: No error", and the ones that wrapped positive + * did not fail at all — they quietly read the wrong place, so a run + * kept its whole working set inside the first 2 GiB while reporting + * the size it was asked for. The default file size is 8 GB, so this + * was the default path, and a 2 GiB hot region flatters a consumer + * SSD enough to make the number worth more than it is. */ + int64_t off = (int64_t)(seed % nrec) * (int64_t)g_rec; int64_t r = waste_pread(fd, buf, g_rec, off); if (r != (int64_t)g_rec) { fprintf(stderr, "short read %lld: %s\n", (long long)r, strerror(errno)); break; } got += r;