From 8355137ed317572f77206780b7540d40fa0c327f Mon Sep 17 00:00:00 2001 From: Alexey Zimarev Date: Wed, 29 Jul 2026 18:58:18 +0200 Subject: [PATCH] docs(dotnet-next): document checkpoint progress without events for filtered $all subscriptions Eventuous/eventuous#560 made the filtered AllStreamSubscription commit a checkpoint at the caught-up transition, establishing the invariant that a caught-up subscription always has a stored checkpoint at or past the head it caught up to. Documents the behavior on the checkpoints concept page and in the KurrentDB subscription options section. Co-Authored-By: Claude Fable 5 --- src/content/docs/dotnet-next/infra/esdb.md | 4 ++++ src/content/docs/dotnet-next/subscriptions/checkpoint.mdx | 6 ++++++ 2 files changed, 10 insertions(+) diff --git a/src/content/docs/dotnet-next/infra/esdb.md b/src/content/docs/dotnet-next/infra/esdb.md index dc7f7c2..806bc25 100644 --- a/src/content/docs/dotnet-next/infra/esdb.md +++ b/src/content/docs/dotnet-next/infra/esdb.md @@ -76,6 +76,10 @@ Subscription options for `AllStreamSubscription` are defined in `AllStreamSubscr | `EventFilter` | Filter for events, if `null`, the subscription will filter out system events. | | `CheckpointInterval` | Interval between checkpoints when event filter is used. Default is `10` events. This interval tells the subscription to report the current checkpoint when the subscription doesn't receive any events for this interval because all the events were filtered out. | +:::note[Checkpoints with sparse filters] +When the event filter excludes events, the server reports its scan position periodically (every `CheckpointInterval` × the filter's max search window scanned events), and the subscription commits that position even though no event reached any handler. In addition, when the subscription catches up with the end of `$all` — on start, and each time it recovers after falling behind — it commits the `$all` head position it observed before subscribing. As a result, a caught-up subscription always has a stored checkpoint at or past the head it caught up to, even on a small or idle store where the filter matched nothing at all. Progress reporting and readiness gates that compare the stored checkpoint with the `$all` head can rely on this: the reported lag of a caught-up subscription converges to zero, and a restart doesn't re-scan the unmatched tail. +::: + #### Checkpoint store `AllStreamSubscription` is a catch-up subscription that is fully managed on the client side (your application), so you need to manage the [checkpoint](../../subscriptions/checkpoint). You can register the checkpoint store using `AddCheckpointStore`, but in that case it will be used for all subscriptions in the application. It might be that your app has multiple subscriptions, and you want to use different checkpoint stores for each of them. In that case, you can register the checkpoint store for each subscription using `UseCheckpointStore` extension of the subscription builder diff --git a/src/content/docs/dotnet-next/subscriptions/checkpoint.mdx b/src/content/docs/dotnet-next/subscriptions/checkpoint.mdx index d0e689c..b08837c 100644 --- a/src/content/docs/dotnet-next/subscriptions/checkpoint.mdx +++ b/src/content/docs/dotnet-next/subscriptions/checkpoint.mdx @@ -151,3 +151,9 @@ On the illustration above, the commit queue has a gap, and event **95** is still ::: As we talk about gaps, you might face a situation when the commit handler has a list of uncommitted checkpoints with gaps, and the application stops. When this happens, some events were already processed, whilst checkpoints for those events remain in-flight. When the application restarts, it loads the checkpoint that points to some position in the stream that is _earlier_ than positions of already processed events. Because of that, some events will be processed by event handlers _again_. Therefore, you need to make sure that your event handlers are _idempotent_, so when the same events are processed again, the result of the processing won't create any undesired side effects. + +## Checkpoint progress without events + +A subscription's checkpoint normally advances when handlers acknowledge processed events. Subscriptions to the global stream with a server-side filter also advance it without delivering anything: the server periodically reports how far it has scanned past filtered-out events, and the subscription commits the reported position through the same commit handler, so the ordering and gap-detection guarantees described above are unaffected. The subscription also commits the position of the global stream's head it observed before subscribing once it catches up (reaches the live edge) — on start, and after every fall-behind recovery. + +This gives filtered global stream subscriptions a useful invariant: once caught up, the stored checkpoint is always at or past the head the subscription caught up to. Catch-up progress reporting and readiness gates that compare the stored checkpoint against the head of the store can rely on it — the lag of a caught-up subscription converges to zero even when its filter matches nothing (for example, a fresh store with a category filter for streams that don't exist yet), and restarts don't re-scan long unmatched stretches.