Add temporal-workflowstreams contrib module#2912
Open
brianstrauch wants to merge 2 commits into
Open
Conversation
A durable, multi-topic pub/sub log hosted inside a workflow, mirroring the workflow streams contrib packages in the Go, Python, and TypeScript SDKs. External publishers send batches via a signal, subscribers long-poll via an update, and a query exposes the current offset; the wire protocol (handler names, JSON envelope field names, base64-of-proto per-item payload encoding) matches the other SDKs exactly for cross-language interop. The workflow side registers a typed listener and supports publisher dedup, ~1 MB poll response paging, truncation, and continue-as-new state carryover. The client side provides a batching publisher with retry and sequence-based exactly-once delivery, and a blocking subscription iterator that follows continue-as-new chains and ends cleanly on terminal workflow states. Like the Go SDK, the core SDK now permits registering signal, update, and query handlers in the __temporal_workflow_stream_ sub-namespace, which is otherwise reserved. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mirrors the CODEOWNERS entries in sdk-go and sdk-python for their workflow streams contrib packages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was changed
Adds a new
temporal-workflowstreamscontrib module: a durable, multi-topic pub/sub log hosted inside a workflow, mirroring the workflow streams contrib packages in the Go, Python, and TypeScript SDKs. All APIs are marked@Experimental.Wire protocol (cross-SDK contract): external publishers send batches via the
__temporal_workflow_stream_publishsignal, subscribers long-poll via the__temporal_workflow_stream_pollupdate, and the__temporal_workflow_stream_offsetquery exposes the current offset. The JSON envelope field names and the per-item payload encoding (base64 of the serializedtemporal.api.common.v1.Payload) match the other SDKs exactly, so a Java publisher or subscriber interoperates with a workflow written in any of them and vice versa.Workflow side:
WorkflowStreamregisters a typed listener (preferably from a@WorkflowInitconstructor) and supports publisher dedup (publisher ID + monotonic sequence), ~1 MB poll response paging, truncation, and continue-as-new state carryover viaWorkflowStreamState.Client side:
WorkflowStreamClient(also constructible from inside an activity) provides a batching background publisher with retry and sequence-based exactly-once delivery, plus a blocking subscription iterator that follows continue-as-new chains, recovers from truncation, and ends cleanly on terminal workflow states.Core SDK change: like the Go SDK (
isWorkflowStreamReservedName), the signal/update/query dispatchers and listener registration now permit handler names in the exact__temporal_workflow_stream_sub-namespace, which is otherwise reserved under the__temporal_prefix. Other__temporal_names remain blocked.Why?
Brings the Java SDK to parity with the other SDKs' experimental workflow streams support, enabling durable event streams whose cost scales with durable batches rather than message count.
Checklist
Closes: n/a
How was this tested:
PayloadWireTest— wire-format round-trips (base64-of-proto, the cross-SDK contract)StreamPublisherTest— publish-path unit tests with an injected signal function (batching, sequence advancement, force flush, drain-on-close, flush timeout)WorkflowStreamTest— workflow-side integration tests against the in-process test server (external publish, dedup, topic-filtered polls, truncation errors,@WorkflowInitconstruction, custom payload converters)SubscribeTest— subscription integration tests (delivery and offset advancement, topic filtering, truncation reset, clean terminal end, continue-as-new follow across runs)WorkflowStreamReservedNameTest— reserved-name exemption works end-to-end; other__temporal_names still rejectedAny docs updates needed: module README included (
contrib/temporal-workflowstreams/README.md)🤖 Generated with Claude Code