Problem
#554 wired the server checkpointReached callback into the filtered AllStreamSubscription, so the stored checkpoint advances across long unmatched stretches. That fix turns out to be necessary but not sufficient: the server only emits a checkpoint message every N scanned events, where N = the checkpoint interval multiplier (AllStreamSubscriptionOptions.CheckpointInterval, default 10) × the filter's max search window (client default 32) — roughly every 320 events read with defaults. Two gaps remain:
- Small stores. A subscription over a store whose entire
$all is shorter than the interval never receives a single checkpoint message during catch-up. If its filter also matches nothing (e.g. a stream-prefix filter for a category that has no streams yet), it never commits anything at all: GetCheckpoint keeps returning null, and the checkpoint store shows no row / position 0 indefinitely.
- Idle tails. Even on large stores, once the subscription passes the last interval boundary (or last matched event) and goes live, a quiet server sends nothing further. The stored checkpoint parks up to interval−1 events below the
$all head for as long as no new events are written.
Consequences:
- a restart re-scans everything since the parked position;
- any consumer that compares the stored checkpoint against the
$all head — catch-up progress reporting, readiness gates that hold work until subscriptions are caught up — sees a phantom, never-closing lag. In the never-committed case the lag reads as 0% forever, which can permanently wedge such a gate on a fresh store: the gate waits for the checkpoint to move, and the checkpoint can only move if events are written past the gate.
Repro sketch: fresh store; write a few dozen events, none matching the filter; start a filtered AllStreamSubscription with a checkpoint store. No checkpoint is ever stored, while an unfiltered subscription alongside it advances to the head. Subscriptions whose filter matched some events park at the last matched event's position instead — same effect, nonzero offset.
Proposed solution
Commit a checkpoint at the caught-up transition, reusing the #554 machinery:
- Observe the client's caught-up notification (
StreamMessage.CaughtUp from the Messages-based SubscribeToAll API; the callback overload currently used doesn't surface it).
- On CaughtUp, route the position through the existing
HandleCheckpointReached(position) path — synthetic payload-less context through the ordered commit machinery, exactly like a server checkpoint message — so ordering guarantees are untouched.
- When the server's CaughtUp message carries no position (older protocol revisions), fall back to a
$all head read taken just before subscribing: everything at or below that position has provably been scanned by the time CaughtUp arrives.
- Re-commit on every later FellBehind → CaughtUp cycle; a monotonic checkpoint store makes that idempotent.
This establishes the invariant "a caught-up subscription always has a stored checkpoint at or past the head it caught up to", closing both gaps: sparse/fresh subscriptions commit on their first (near-instant) catch-up, and idle tails get committed at the live switch. It also composes with StartFrom = Latest (CaughtUp fires immediately after subscribing from End; committing the head is correct there too).
StreamSubscription doesn't strictly need this (it isn't server-filtered and commits per received event), but the same treatment would be harmless and symmetric.
Regression test note
Any test for this must keep the store below the checkpoint interval (or use a filter matching nothing and a quiet store) — writing a few hundred events first masks the bug via the interval mechanism.
Problem
#554 wired the server
checkpointReachedcallback into the filteredAllStreamSubscription, so the stored checkpoint advances across long unmatched stretches. That fix turns out to be necessary but not sufficient: the server only emits a checkpoint message every N scanned events, where N = the checkpoint interval multiplier (AllStreamSubscriptionOptions.CheckpointInterval, default 10) × the filter's max search window (client default 32) — roughly every 320 events read with defaults. Two gaps remain:$allis shorter than the interval never receives a single checkpoint message during catch-up. If its filter also matches nothing (e.g. a stream-prefix filter for a category that has no streams yet), it never commits anything at all:GetCheckpointkeeps returning null, and the checkpoint store shows no row / position 0 indefinitely.$allhead for as long as no new events are written.Consequences:
$allhead — catch-up progress reporting, readiness gates that hold work until subscriptions are caught up — sees a phantom, never-closing lag. In the never-committed case the lag reads as 0% forever, which can permanently wedge such a gate on a fresh store: the gate waits for the checkpoint to move, and the checkpoint can only move if events are written past the gate.Repro sketch: fresh store; write a few dozen events, none matching the filter; start a filtered
AllStreamSubscriptionwith a checkpoint store. No checkpoint is ever stored, while an unfiltered subscription alongside it advances to the head. Subscriptions whose filter matched some events park at the last matched event's position instead — same effect, nonzero offset.Proposed solution
Commit a checkpoint at the caught-up transition, reusing the #554 machinery:
StreamMessage.CaughtUpfrom theMessages-basedSubscribeToAllAPI; the callback overload currently used doesn't surface it).HandleCheckpointReached(position)path — synthetic payload-less context through the ordered commit machinery, exactly like a server checkpoint message — so ordering guarantees are untouched.$allhead read taken just before subscribing: everything at or below that position has provably been scanned by the time CaughtUp arrives.This establishes the invariant "a caught-up subscription always has a stored checkpoint at or past the head it caught up to", closing both gaps: sparse/fresh subscriptions commit on their first (near-instant) catch-up, and idle tails get committed at the live switch. It also composes with
StartFrom = Latest(CaughtUp fires immediately after subscribing from End; committing the head is correct there too).StreamSubscriptiondoesn't strictly need this (it isn't server-filtered and commits per received event), but the same treatment would be harmless and symmetric.Regression test note
Any test for this must keep the store below the checkpoint interval (or use a filter matching nothing and a quiet store) — writing a few hundred events first masks the bug via the interval mechanism.