Summary
pairedEvalueSequence (v0.144.6) cannot cross 1/alpha = 20 before t = 24 pairs even on perfect data — every delta at the bound. A consumer with a short spend cap gets a sequence that is undecidable by construction, while the anytime-valid framing suggests it can stop early. This burned a 3-pair, ~90M-token generation in discovery-lab before simulation exposed it.
Repro (v0.144.6)
import { pairedEvalueSequence } from '@tangle-network/agent-eval'
// perfect data: every paired delta at the declared bound
for (const n of [6, 24]) {
const r = pairedEvalueSequence(Array(n).fill(5), { alpha: 0.05, bound: 5 })
console.log(n, r.steps.at(-1).evalue.toFixed(4), r.decisionFiredAt)
}
// 6 -> e = 1.2560, decisionFiredAt = null
// 24 -> first crossing at t = 24 (alpha = 0.05; scale-invariant in the bound)
Realistic data is worse: deltas at 60% of the bound first cross at t = 39.
Asks, in order of preference
- Expose the horizon: a
sequentialCrossingHorizon({ alpha, bound }) (or a field on the result) that names the minimum t at which ANY data can cross 1/alpha. Consumers must be able to gate a design on reachability the same way minimumPairsForPairedDeltaTest gates fixed-n designs. Discovery-lab now computes this by simulating Array(n).fill(bound) — it belongs in the package next to the other floors.
- Document the conservatism: if the current mixture bet is intentional, the doc comment should state the perfect-data horizon so a 6-pair cap is visibly absurd at authoring time.
- Consider a more aggressive bet option: t = 24 on all-at-bound data is far from Kelly-optimal growth for that stream; an opt-in aggressive bet would make short sequential designs usable at all.
Context
Found while closing discovery-lab generation 4: cap 6 pairs, observed deltas [2, 5, 1] on bound 5, final e = 1.0103. The lab's design gate now simulates the best case and refuses caps below the horizon; the floor should come from this package rather than be re-derived downstream (same principle as minimumPairsForPairedDeltaTest).
Summary
pairedEvalueSequence(v0.144.6) cannot cross1/alpha = 20before t = 24 pairs even on perfect data — every delta at the bound. A consumer with a short spend cap gets a sequence that is undecidable by construction, while the anytime-valid framing suggests it can stop early. This burned a 3-pair, ~90M-token generation in discovery-lab before simulation exposed it.Repro (v0.144.6)
Realistic data is worse: deltas at 60% of the bound first cross at t = 39.
Asks, in order of preference
sequentialCrossingHorizon({ alpha, bound })(or a field on the result) that names the minimum t at which ANY data can cross1/alpha. Consumers must be able to gate a design on reachability the same wayminimumPairsForPairedDeltaTestgates fixed-n designs. Discovery-lab now computes this by simulatingArray(n).fill(bound)— it belongs in the package next to the other floors.Context
Found while closing discovery-lab generation 4: cap 6 pairs, observed deltas [2, 5, 1] on bound 5, final e = 1.0103. The lab's design gate now simulates the best case and refuses caps below the horizon; the floor should come from this package rather than be re-derived downstream (same principle as
minimumPairsForPairedDeltaTest).