Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/content/docs/dotnet-next/infra/esdb.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ Subscription options for `AllStreamSubscription` are defined in `AllStreamSubscr
| `EventFilter` | Filter for events, if `null`, the subscription will filter out system events. |
| `CheckpointInterval` | Interval between checkpoints when event filter is used. Default is `10` events. This interval tells the subscription to report the current checkpoint when the subscription doesn't receive any events for this interval because all the events were filtered out. |

:::note[Checkpoints with sparse filters]
When the event filter excludes events, the server reports its scan position periodically (every `CheckpointInterval` × the filter's max search window scanned events), and the subscription commits that position even though no event reached any handler. In addition, when the subscription catches up with the end of `$all` — on start, and each time it recovers after falling behind — it commits the `$all` head position it observed before subscribing. As a result, a caught-up subscription always has a stored checkpoint at or past the head it caught up to, even on a small or idle store where the filter matched nothing at all. Progress reporting and readiness gates that compare the stored checkpoint with the `$all` head can rely on this: the reported lag of a caught-up subscription converges to zero, and a restart doesn't re-scan the unmatched tail.
:::

#### Checkpoint store

`AllStreamSubscription` is a catch-up subscription that is fully managed on the client side (your application), so you need to manage the [checkpoint](../../subscriptions/checkpoint). You can register the checkpoint store using `AddCheckpointStore<T>`, but in that case it will be used for all subscriptions in the application. It might be that your app has multiple subscriptions, and you want to use different checkpoint stores for each of them. In that case, you can register the checkpoint store for each subscription using `UseCheckpointStore<T>` extension of the subscription builder
Expand Down
6 changes: 6 additions & 0 deletions src/content/docs/dotnet-next/subscriptions/checkpoint.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -151,3 +151,9 @@ On the illustration above, the commit queue has a gap, and event **95** is still
:::

As we talk about gaps, you might face a situation when the commit handler has a list of uncommitted checkpoints with gaps, and the application stops. When this happens, some events were already processed, whilst checkpoints for those events remain in-flight. When the application restarts, it loads the checkpoint that points to some position in the stream that is _earlier_ than positions of already processed events. Because of that, some events will be processed by event handlers _again_. Therefore, you need to make sure that your event handlers are _idempotent_, so when the same events are processed again, the result of the processing won't create any undesired side effects.

## Checkpoint progress without events

A subscription's checkpoint normally advances when handlers acknowledge processed events. Subscriptions to the global stream with a server-side filter also advance it without delivering anything: the server periodically reports how far it has scanned past filtered-out events, and the subscription commits the reported position through the same commit handler, so the ordering and gap-detection guarantees described above are unaffected. The subscription also commits the position of the global stream's head it observed before subscribing once it catches up (reaches the live edge) — on start, and after every fall-behind recovery.

This gives filtered global stream subscriptions a useful invariant: once caught up, the stored checkpoint is always at or past the head the subscription caught up to. Catch-up progress reporting and readiness gates that compare the stored checkpoint against the head of the store can rely on it — the lag of a caught-up subscription converges to zero even when its filter matches nothing (for example, a fresh store with a category filter for streams that don't exist yet), and restarts don't re-scan long unmatched stretches.