Pin DDSketch's value-to-bucket mapping across architectures

[mattlorimor]

Pins the bucket a value falls in, for 28 values across two accuracies, read straight out of the payload.

Why this and not the ILogB change

The ILogB idea I floated doesn't survive contact. value = m × 2^e gives log2(value) = e + log2(m), but log2(m) is still a libm call and still isn't correctly rounded — it improves the conditioning and delivers no determinism. Getting determinism needs the mantissa's log computed from IEEE-mandated operations only, i.e. a polynomial mapping.

That's a bigger deal than it sounds, because bucket indices are persisted:

i32     the index of its first bucket
bytes   a u64 count per bucket, from that index upward

The mapping isn't stored, but it's a format contract. Change it and every existing payload's indices mean something new, with nothing in the file to say so. Merge has the same exposure across mixed-architecture nodes. So a deterministic mapping costs a format version and a dual-read path.

What the exposure actually is

Measured rather than argued:

accuracy	P(a value lands close enough to a boundary for the last bit to decide)
0.001	~1 in 8.6 × 10⁹
0.01	~1 in 6.9 × 10¹⁰
0.1	~1 in 1.1 × 10¹²

Over 2M log-uniform values spanning 1µs–100s, the closest any got to a boundary was ~10⁵ ulps. And perturbing Math.Log by a full ulp — more than a libm difference plausibly is — moves none of the 28 pinned values. The sole exception is 1.0, and only because log(1.0) is exactly 0.0, so a nudge shifts it off a boundary it sits exactly on; IEEE mandates that zero on every platform, so it doesn't move in reality either.

Conclusion: not worth a format version. The mapping stays as it is.

What this adds instead

• TestBucketIndicesHaveNotMovedAtOnePercent — 21 values, 1µs to 10s
• TestBucketIndicesHaveNotMovedAtATenthOfAPercent — 7 values where |q| is largest and drift has the most room
• TestTheNumberBeingReadIsTheBucketIndex — anchors needing no logarithm (1.0→0, 1.01→1, 0.97→−1), each well inside its bucket, so they hold whatever the last bit does. Also covers the signed u32 round trip, since indices go negative below one.

Expected indices were derived from the paper's definition rather than copied from this library's output, so a wrong rounding would surface instead of being enshrined. That was done on one machine and proves nothing about others by itself — what checks architectures is this file running on x64 Linux, x64 Windows and arm64 macOS (macos-latest is macos-26-arm64).

Mutation

• Ceiling → Floor: all 3 killed.
• Math.Log(value) perturbed by +1 ulp: all 3 killed, but only via the 1.0 case — recorded in the file, since it's the honest limit of what this pins. It guards the mapping against being changed, not against drifting; drift that small demonstrably doesn't reach these values.

929 tests pass, clean -warnaserror build from scratch.

---

Added: pinning the divisor

Working out the actual odds exposed a gap in the above. The value corpus guards the wrong channel.

Channel A — per-value. Two libms disagree on log(v). Measured against a 60-digit correctly-rounded reference: 0.009% of arguments differ, always by exactly 1 ulp. Propagated into the quotient that's 1 in 2.5 × 10¹⁸ (a=0.01). Zero bucket flips in 300k values.

Channel B — the divisor. gamma is (1+a)/(1-a): correctly-rounded arithmetic, identical everywhere. log(gamma) is not. Every index divides by that one number, so an ulp shifts every quotient at once, amplified by |q| (mean 288 at a=0.01, 2,880 at a=0.001) — 1 in 2 × 10¹³, five orders of magnitude worse, and correlated across the whole sketch instead of scattered.

TestTheDivisorIsTheSameNumberOnEveryPlatform pins logGamma to the bit for seven accuracies. The pinned values are the correctly rounded logarithms, computed to sixty digits and rounded once — not this machine's output — so a platform whose libm is a bit out fails rather than having its answer enshrined.

Mutation (logGamma += 1 ulp) shows why it was needed:

test	result
TestTheDivisorIsTheSameNumberOnEveryPlatform	failed — caught it
the 3 value-corpus tests	passed — missed it entirely

Which is the predicted behaviour: 28 values at 5×10⁻¹⁴ each notice a divisor divergence about one time in 10¹². Without this, the corpus would pass happily on a platform where every sketch was quietly bucketing differently.

930 tests pass, clean -warnaserror build from scratch.