-
Notifications
You must be signed in to change notification settings - Fork 10
feat(hook): Deliver events to integrations #607
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
prathshenoy
wants to merge
1
commit into
main
Choose a base branch
from
prath.shenoy/hook-dispatch
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| load("@rules_go//go:def.bzl", "go_library") | ||
|
|
||
| go_library( | ||
| name = "go_default_library", | ||
| srcs = ["hook.go"], | ||
| importpath = "github.com/uber/submitqueue/platform/extension/hook", | ||
| visibility = ["//visibility:public"], | ||
| deps = ["//api/base/hook:go_default_library"], | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| # Hook | ||
|
|
||
| Vendor-agnostic interface for fire-and-forget side effects run in response to pipeline lifecycle events: warehouse exports, code-host comments, notifications, audit trails. See [the hooks framework RFC](../../../doc/rfc/hook-framework.md) for the design and [`api/base/hook`](../../../api/base/hook) for the event contract. | ||
|
|
||
| ## Interface | ||
|
|
||
| ### Hook | ||
|
|
||
| Handles one lifecycle event. `Name` identifies it in logs, metrics, and failure attribution. | ||
|
|
||
| Four obligations, all of them consequences of running behind an at-least-once queue: | ||
|
|
||
| - **Idempotent on the event id.** The same event may arrive more than once, including after a successful `Handle`. The id is derived from the transition, so a redelivery carries the id the first delivery did. | ||
| - **Return nil to ignore an event.** There is no filter or subscription API. A hook that does not care about a type returns nil and costs nothing; routing can become a wiring decorator if it ever pays for itself. | ||
| - **Return plain errors.** Classification is the consumer's job. An error must mean the side effect did not happen — reporting failure for work that succeeded turns at-least-once delivery into repeated duplicate effects. | ||
| - **Never write pipeline state.** A hook's outcome is invisible to the pipeline, which is exactly what makes it unable to affect the transition that triggered it. | ||
|
|
||
| ### Hooks | ||
|
|
||
| Resolves the hooks that run for an event. The controller in [`platform/hook`](../../hook) asks it once per delivery and runs everything it returns; returning none is ordinary and means nothing this host wired cares about the event. | ||
|
|
||
| `For` takes the event rather than a queue name because the envelope carries no queue. Which scope selects hooks differs per domain — queue, source, event type — and only the host that publishes the payload can read a queue out of it, so the choice belongs to the resolver. Resolution runs on every delivery and cannot fail: an integration that cannot be reached is a `Handle` error, not an absent hook. | ||
|
|
||
| ## Wiring | ||
|
|
||
| There is no `Config` and no `Factory` here. Selection is the resolver's job, and the resolver is built in the wiring layer — the only place that knows the full set of queues and the integrations wired for each. The host constructs its `Hooks` and hands it to the controller in [`platform/hook`](../../hook), which owns the consumer side: decode, validate, resolve, invoke. | ||
|
|
||
| Two queues in one host can point at different providers and want different integrations, which is why hooks are resolved per event rather than fixed per deployment. | ||
|
|
||
| ## Implementations | ||
|
|
||
| - **`noop/`** — accepts every event and does nothing. A placeholder for a host that wants the stage registered before it has any integration; a resolver that returns no hooks does the same thing. | ||
|
|
||
| A sink that serves several domains is one implementation wired into each domain's host, not one implementation per domain. | ||
|
|
||
| ## Implementing a Hook | ||
|
|
||
| 1. Create `platform/extension/hook/{name}/` for a hook reusable across domains, or `{domain}/extension/hook/{name}/` for one that is domain-specific. | ||
| 2. Implement `Handle` and `Name`, keying any deduplication on `event.GetId()`. | ||
| 3. Decide per event `type` what to do, and return nil for the types you ignore. | ||
| 4. Return it from the host's `Hooks` resolver for the events it should run on. | ||
|
|
||
| Every hook the resolver returns for an event shares one consumer and therefore one retry budget: one chronically failing integration eventually dead-letters events the others handled fine. See [`platform/hook`](../../hook) before wiring several. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| // Copyright (c) 2026 Uber Technologies, Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| // Package hook defines the contract for a hook: a pluggable side effect run in | ||
| // response to a pipeline lifecycle event. Warehouse exports, code-host comments, | ||
| // notifications, and audit trails are all hooks. | ||
| // | ||
| // Which hooks run is a property of the event rather than of the deployment: two | ||
| // queues in one host can point at different providers and want different | ||
| // integrations. A host therefore supplies a Hooks resolver, and the controller | ||
| // in platform/hook asks it once per event. | ||
| // | ||
| // Hooks run behind a durable queue, never inline in the pipeline, so a slow or | ||
| // failing integration cannot stall or fail the work that triggered it. | ||
| package hook | ||
|
|
||
| //go:generate mockgen -source=hook.go -destination=mock/hook_mock.go -package=mock | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| basehook "github.com/uber/submitqueue/api/base/hook" | ||
| ) | ||
|
|
||
| // Hook performs a side effect in response to a lifecycle event. | ||
| type Hook interface { | ||
| // Handle performs the side effect for event. | ||
| // | ||
| // Delivery is at-least-once, so the same event — identical id — may arrive | ||
| // more than once, including after a successful Handle. Implementations must | ||
| // be idempotent on the event id. | ||
| // | ||
| // Returning nil means "done with this event", which is also how a hook | ||
| // ignores one: there is no filter or subscription API, because a hook that | ||
| // does not care about a type simply returns nil, and routing can be added as | ||
| // a wiring decorator if it ever pays for itself. | ||
| // | ||
| // Returning an error retries the event and, past the retry budget, | ||
| // dead-letters it. Return plain errors; classification is the consumer's | ||
| // job. An error must mean the side effect did not happen — reporting failure | ||
| // for work that succeeded turns at-least-once into repeated duplicate | ||
| // effects. | ||
| // | ||
| // A hook must never write pipeline state. Its outcome is invisible to the | ||
| // pipeline by design: that is what makes the side effect unable to affect | ||
| // the transition that triggered it. | ||
| Handle(ctx context.Context, event *basehook.HookEvent) error | ||
|
|
||
| // Name identifies the hook in logs, metrics, and the failure attribution | ||
| // the controller reports. Stable and unique among the hooks a host wires. | ||
| Name() string | ||
| } | ||
|
|
||
| // Hooks resolves the hooks that run for an event. | ||
| type Hooks interface { | ||
| // For returns the hooks to run for event, in the order they should run. | ||
| // Returning none is an ordinary outcome: it means nothing this host wired | ||
| // is interested in the event. | ||
| // | ||
| // It takes the event rather than a queue name because the envelope carries | ||
| // no queue. Which scope selects hooks differs per domain — queue, source, | ||
| // event type — and only the host that publishes the payload can read a | ||
| // queue out of it, so the choice belongs to the resolver. | ||
| // | ||
| // Called on every delivery, so resolution must be cheap and must not fail: | ||
| // an integration that cannot be reached is a Handle error, not an absent | ||
| // hook. | ||
| For(event *basehook.HookEvent) []Hook | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| load("@rules_go//go:def.bzl", "go_library") | ||
|
|
||
| go_library( | ||
| name = "go_default_library", | ||
| srcs = ["hook_mock.go"], | ||
| importpath = "github.com/uber/submitqueue/platform/extension/hook/mock", | ||
| visibility = ["//visibility:public"], | ||
| deps = [ | ||
| "//api/base/hook:go_default_library", | ||
| "//platform/extension/hook:go_default_library", | ||
| "@org_uber_go_mock//gomock:go_default_library", | ||
| ], | ||
| ) |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| load("@rules_go//go:def.bzl", "go_library", "go_test") | ||
|
|
||
| go_library( | ||
| name = "go_default_library", | ||
| srcs = ["hook.go"], | ||
| importpath = "github.com/uber/submitqueue/platform/extension/hook/noop", | ||
| visibility = ["//visibility:public"], | ||
| deps = [ | ||
| "//api/base/hook:go_default_library", | ||
| "//platform/extension/hook:go_default_library", | ||
| ], | ||
| ) | ||
|
|
||
| go_test( | ||
| name = "go_default_test", | ||
| srcs = ["hook_test.go"], | ||
| embed = [":go_default_library"], | ||
| deps = [ | ||
| "//api/base/hook:go_default_library", | ||
| "@com_github_stretchr_testify//require:go_default_library", | ||
| ], | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| // Copyright (c) 2026 Uber Technologies, Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| // Package noop provides a hook.Hook that accepts every event and does nothing. | ||
| // It is a placeholder for a host that wants the stage registered before it has | ||
| // any integration — a resolver returning no hooks does the same thing. Either | ||
| // way events are still published, consumed, and acked, so turning a real hook on | ||
| // later changes only what happens to the event, not whether the seam works. | ||
| package noop | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| basehook "github.com/uber/submitqueue/api/base/hook" | ||
| "github.com/uber/submitqueue/platform/extension/hook" | ||
| ) | ||
|
|
||
| // Verify interface compliance at compile time. | ||
| var _ hook.Hook = Hook{} | ||
|
|
||
| // Hook is a hook that discards every event. | ||
| type Hook struct{} | ||
|
|
||
| // New returns a no-op Hook. | ||
| func New() Hook { | ||
| return Hook{} | ||
| } | ||
|
|
||
| // Handle implements hook.Hook. The event is discarded. | ||
| func (Hook) Handle(context.Context, *basehook.HookEvent) error { return nil } | ||
|
|
||
| // Name implements hook.Hook. | ||
| func (Hook) Name() string { return "noop" } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
my general question here is why is it an extension vs a plain consumer who gets hooks events from wire and implements consumer controller? why it needs to be an interface?