Skip to content

Adding large payload support for the standalone SDK#280

Open
bachuv wants to merge 5 commits intomainfrom
vabachu/java-large-payloads
Open

Adding large payload support for the standalone SDK#280
bachuv wants to merge 5 commits intomainfrom
vabachu/java-large-payloads

Conversation

@bachuv
Copy link
Copy Markdown
Contributor

@bachuv bachuv commented Apr 13, 2026

Issue describing the changes in this PR

Adding large payload support for the standalone SDK

resolves #issue_for_this_pr

Pull request checklist

  • My changes do not require documentation changes
    • Otherwise: Documentation issue linked to PR
  • My changes are added to the CHANGELOG.md
  • I have added all required tests (Unit tests, E2E tests)

@bachuv bachuv requested a review from a team as a code owner April 13, 2026 16:11
Copilot AI review requested due to automatic review settings April 13, 2026 16:11
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds “large payload” support to the standalone Durable Task Java SDK by introducing an Azure Blob Storage–backed payload externalization module and wiring gRPC interceptors + orchestrator-response chunking into the core client/worker to avoid gRPC message size limits.

Changes:

  • Introduces new :azure-blob-payloads module implementing payload externalization/resolution via a gRPC ClientInterceptor and an Azure Blob–backed PayloadStore.
  • Adds gRPC interceptor support to DurableTaskGrpcClientBuilder/DurableTaskGrpcWorkerBuilder, plus worker capability announcement and orchestrator completion chunking.
  • Adds a runnable sample and unit/integration tests covering token handling, interceptor behavior, and chunking behavior.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
settings.gradle Adds the new :azure-blob-payloads Gradle module to the build.
samples/src/main/java/io/durabletask/samples/LargePayloadSample.java New sample demonstrating end-to-end large payload externalization with DTS + Azurite.
samples/build.gradle Adds a runLargePayloadSample task and depends on :azure-blob-payloads.
internal/durabletask-protobuf/protos/orchestrator_service.proto Updates protobuf contract (tags, rewind action, purge timeout).
internal/durabletask-protobuf/PROTO_SOURCE_COMMIT_HASH Updates upstream proto source commit hash.
client/src/test/java/com/microsoft/durabletask/OrchestratorChunkingTest.java Adds unit tests for worker chunking + action-size validation + capability announcement.
client/src/main/java/com/microsoft/durabletask/DurableTaskGrpcWorkerBuilder.java Adds interceptor registration, LP capability flag, and configurable chunk size.
client/src/main/java/com/microsoft/durabletask/DurableTaskGrpcWorker.java Applies interceptors, announces LP capability, and implements orchestrator-response chunking/validation.
client/src/main/java/com/microsoft/durabletask/DurableTaskGrpcClientBuilder.java Adds interceptor registration support to the client builder.
client/src/main/java/com/microsoft/durabletask/DurableTaskGrpcClient.java Applies registered interceptors to the client channel.
client/build.gradle Adds grpc-inprocess for new in-process gRPC unit tests.
azure-blob-payloads/src/main/java/com/microsoft/durabletask/azureblobpayloads/PayloadStore.java Introduces PayloadStore abstraction for out-of-band payload storage.
azure-blob-payloads/src/main/java/com/microsoft/durabletask/azureblobpayloads/PayloadStorageException.java Defines exception type for permanent storage failures.
azure-blob-payloads/src/main/java/com/microsoft/durabletask/azureblobpayloads/LargePayloadWorkerExtensions.java Adds worker-side helper methods to enable externalized payloads + capability flag.
azure-blob-payloads/src/main/java/com/microsoft/durabletask/azureblobpayloads/LargePayloadStorageOptions.java Adds configuration options (threshold/max/container/auth/compression).
azure-blob-payloads/src/main/java/com/microsoft/durabletask/azureblobpayloads/LargePayloadInterceptor.java Implements interceptor that externalizes outbound payloads and resolves inbound tokens.
azure-blob-payloads/src/main/java/com/microsoft/durabletask/azureblobpayloads/LargePayloadClientExtensions.java Adds client-side helper methods to enable externalized payloads.
azure-blob-payloads/src/main/java/com/microsoft/durabletask/azureblobpayloads/BlobPayloadStore.java Implements Azure Blob Storage payload store with optional gzip compression.
azure-blob-payloads/src/test/java/com/microsoft/durabletask/azureblobpayloads/PayloadTokenTest.java Unit tests for token encode/decode and token detection.
azure-blob-payloads/src/test/java/com/microsoft/durabletask/azureblobpayloads/LargePayloadStorageOptionsTest.java Unit tests for options defaults and validation behavior.
azure-blob-payloads/src/test/java/com/microsoft/durabletask/azureblobpayloads/LargePayloadInterceptorTest.java Unit tests for request externalization + response resolution across message types.
azure-blob-payloads/src/test/java/com/microsoft/durabletask/azureblobpayloads/LargePayloadIntegrationTest.java Integration tests requiring DTS emulator + Azurite for end-to-end validation.
azure-blob-payloads/src/test/java/com/microsoft/durabletask/azureblobpayloads/BlobPayloadStoreTest.java Unit tests for blob upload/download/compression behavior using mocks.
azure-blob-payloads/spotbugs-exclude.xml SpotBugs exclusions for the new module.
azure-blob-payloads/build.gradle Build/test/spotbugs config and dependencies for the new module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants