fix: restore Sentry scope isolation when OpenTelemetry is enabled (AI-3346) - #16
Merged
Conversation
`initializeSentry` passes `skipOpenTelemetrySetup: config.enableOtel`, which skips `initOpenTelemetry()`. That is the only place @sentry/node installs `SentryContextManager`, while @sentry/node-core's `init` installs the OpenTelemetry async context strategy unconditionally. Sentry's scope functions were therefore backed by the OTel context with nothing writing scopes onto it, so `withIsolationScope` and `getCurrentScope` resolved to the process-global scope and tags, user identity and transaction names crossed between concurrent requests and background jobs. `provider.register()` now installs `SentryContextManager`. The same skipped setup also leaves `client.traceProvider` unassigned, which is the only handle `NodeClient.flush()` and `.close()` have on the span processors, so a graceful shutdown discarded whatever the OTLP batch was holding. `attachTraceProviderForShutdownFlush` supplies it, bounded by `forceFlushTimeoutMillis` and with span export failures contained, because `BasicTracerProvider.forceFlush()` rejects with an array of errors and `NodeClient.flush()` awaits it uncaught -- an unreachable collector would otherwise abort the client's own event flush and drop buffered errors. It is skipped when the client already has a provider, so a consumer that completed its own OpenTelemetry setup keeps the one that actually receives spans. `warnIfScopeIsolationInactive` reports when isolation is inactive anyway. `setGlobalContextManager` refuses to replace an existing manager and reports the refusal only on OpenTelemetry's diag channel; neither SDK validator catches that, since `validateOpenTelemetrySetup` early-returns in non-debug builds and `openTelemetrySetupCheck` records elements at construction rather than at successful registration. Refs AI-3346 Co-Authored-By: Claude <noreply@anthropic.com>
andybevan-scope3
marked this pull request as ready for review
July 29, 2026 21:53
benminer
approved these changes
Jul 29, 2026
benminer
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed the isolation and shutdown changes. The OTLP shutdown path is now bounded and covered by a close-path regression test.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes cross-request leakage of tags, user identity and transaction names. Live in prod since 2026-05-26 and measured leaking customer and user identity between requests.
Linear: AI-3346
Root cause
initializeSentrypassesskipOpenTelemetrySetup: config.enableOtel, which skipsinitOpenTelemetry()(@sentry/node/build/cjs/sdk/index.js:57). That is the only place the SDK installsSentryContextManager. Meanwhile@sentry/node-core'sinitcallssetOpenTelemetryContextAsyncContextStrategy()unconditionally (sdk/index.js:94).So Sentry's scope functions were backed by the OTel context, with nothing ever writing Sentry's scopes onto that context. Every
withIsolationScope/getCurrentScoperesolved to the process-global default scope.What this changes
provider.register()installsSentryContextManager. This is the isolation fix.attachTraceProviderForShutdownFlush(). The same skipped setup leavesclient.traceProviderunassigned, and that is the only handleNodeClient.flush()/.close()have on the span processors, so graceful shutdown was discarding whatever the OTLP batch held.warnIfScopeIsolationInactive().setGlobalContextManagerrefuses to replace an existing manager and reports the refusal only on OTel's diag channel.Why (2) is not a one-line assignment
The naive
client.traceProvider = provideris a regression, which is why it carries two extra pieces:BasicTracerProvider.forceFlush()rejects with an array of errors if any processor fails or times out (sdk-trace-base/BasicTracerProvider.js:74-85), andNodeClient.flush()awaits it uncaught. With an unreachable collector at SIGTERM,await Sentry.close(2000)would reject beforesuper.close()ran, silently discarding buffered Sentry error events. Export failures are now contained.forceFlushTimeoutMillisis 30s, matching this provider'sexportTimeoutMillis, and the caller'sflush(timeout)is not passed through. That can outlast a K8s termination grace period. Now bounded to 500ms, matching what Sentry's ownsetupOtelpicks.client.traceProvideris already set: if a consumer completed its own OTel setup first, ourregister()was silently refused and our provider will never see a span, so overwriting theirs would point shutdown at a dead provider.Why the guard is hand-rolled
Both SDK validators were tried and read from installed source.
validateOpenTelemetrySetup()early-returns unlessDEBUG_BUILDand reports viacore.debug.error, silent unlessdebug: true.openTelemetrySetupCheck()callssetIsSetup()from element constructors, so it reportsSentryContextManageras present merely because one was constructed, and keepssetupElementsper module instance. The guard compares scope identity instead, which is exact.Verification
Every assertion was checked by removing the fix and confirming the right tests fail, on a clean
npm ciinstall with Node 24.contextManagerremovedtraceProviderassignment removedpromise rejected "[ …(1) ]"config.tspre-existing)settles the client flush when span export failsruns in ~510ms, which is the 500ms bound doing its job rather than the 30s default.Incidental:
package-lock.jsonwas stale on main (still@scope3data/observability-jsat2.0.0) and is regenerated here.@opentelemetry/context-async-hooksmoves to an explicit devDependency; the guard test needs the scope-unaware manager thatSentryContextManagerwraps, and relying on it transitively would break on a dedupe.Decisions I want a second opinion on
2.2.0. Strict semver says2.1.1. Minor argues consumers see a real behavior change. There is a case this is closer to a major:SentryContextManager.with()clones the current scope, so aSentry.setTag()inside astartSpancallback now lands on the fork and no longer persists after it. That was the intended Sentry semantic all along, but it is a behavior change. Note anyone pinned~2.1.0gets no PII-leak fix at all under a minor.console.errorvs throwing in the guard, givensendDefaultPii: trueis set. Log-and-continue means a detected isolation failure still serves traffic.Not in this PR
Real, none required for AI-3346. Detail in
.context/todos.md.buildTracesSamplerthrows on array attribute values.rawRoute.replaceon anhttp.targetof['/a']givesTypeError, from insideSentrySampler.shouldSample, on every span. Found while probing the flush path.defaultResource(), soOTEL_RESOURCE_ATTRIBUTESand alltelemetry.sdk.*are missing from exported spans.resetForTesting()does not clear the OTel globals, so a documented reset-then-init()is silently refused, and the guard is blind to it. The new test hand-rollsclearOtelGlobals()for this reason.client.asyncLocalStorageLookupnever assigned (latent; nothing in @sentry 10.34 reads it).setGlobalTracerProvider/setGlobalPropagatorfail just as silently and their booleans are discarded.Before the agentic-api bump
Audit every
withIsolationScopecall site first. Those wraps are no-ops today; once this ships, tags set inside one stop being visible outside it. Any capture currently reading a tag it never set (and getting the right answer via the leak) starts reading nothing. PR #6738 there was written on the false premise that adding wraps would help, when the wrap was the broken thing.Sentry issue grouping will also shift, since
transactionis wrong on many events today, so expect a wave of apparently-new issues that are regrouped existing ones. Announce before it lands.🤖 Generated with Claude Code