fix(traces): always open vcon_processing root span, link if upstream context present#181
Merged
pavanputhra merged 2 commits intoMay 21, 2026
Merged
Conversation
… global Follow-up to #178. The fork-safe init landed, but SignOz queries still showed only one worker's value (peak inflight 128 across 4 worker processes instead of the expected ~512). Root cause: the user-provided OTel resource (``host.name``, ``service.instance.id``) never reached the metrics backend. OpenTelemetry Python's ``metrics.set_meter_provider()`` is single-call. When ``opentelemetry-instrumentation`` (auto-instrumentation) is active in the process, it registers a default MeterProvider early in startup — before this module's lazy ``_init_otel_metrics()`` runs. Our subsequent ``set_meter_provider(our_provider)`` call is silently ignored, and ``metrics.get_meter(__name__)`` returns the global proxy bound to the auto-instrumentation's provider — whose resource we don't control. The visible CH symptom: ``resource_attrs`` on every conserver.* metric only contains ``service.name`` + ``telemetry.sdk.*`` + ``telemetry.auto.version``; ``host.name`` and ``service.instance.id`` that this module sets are dropped. All four worker processes collapse onto a single fingerprint, last-write-wins. Fix: bind ``meter`` to OUR provider directly via ``provider.get_meter(__name__)``, never touching the global. Our provider has the correct resource and its own export pipeline to the OTel collector. Auto-instrumentation metrics continue to flow through their own pipeline independently — both arrive at the collector. Tests updated to assert ``set_meter_provider`` is NOT called and the ``meter`` global is bound to the provider's own meter, not the global proxy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…context present
Previously the vcon_processing.<chain> span was only created when
self.context carried an upstream trace context (extracted from Redis
under context:{ingress}:{vcon_uuid}). When the producer didn't store
a context — e.g. an adapter that ingests directly to the ingress list
without OTel instrumentation — _create_span_from_context short-
circuited to None and every link.* / storage.* span emitted during
the chain run became its own root trace, defeating the
trace-per-vcon model.
Refactor _create_span_from_context to always return a span context
manager. A span link to the producer's trace is attached only when
trace_id and span_id parse to non-zero values; otherwise the span
opens without links. Trace structure is preserved either way, and
the link semantics still light up automatically once producers (e.g.
the BDS adapter) begin propagating context via store_context_*.
Net: -27 lines (139 changed, 56+/83-). Run-method setup collapses
from a 25-line if/else dance to two lines; the POC trace-id
verification log goes away.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_create_span_from_contextso it always returns a span context manager — neverNone.Linkto the producer's trace is attached only whenself.contextcarries a validtrace_id/span_idpair. When no upstream context is propagated, the span is created with no links so childlink.*/storage.*spans still nest under a single root.run()setup collapses from a 25-line if/else dance to two lines; the POC trace-id verification log goes away.Why
Before this change, if the producer didn't
store_context_*to Redis before enqueueing — e.g. an adapter that ingests directly without OTel —_create_span_from_contextshort-circuited and everylink.*/storage.*span became its own root trace. The trace-per-vCon model is now preserved regardless of producer instrumentation, and the upstream link still lights up automatically once producers do propagate context.Tests
27 directly relevant tests pass locally (
common/tests/lib/test_context_utils.py,test_inflight_counter.py,test_parallel_processing.py,test_otel_fork_safe_init.py). The unrelated link-metrics failures in the local venv are pre-existingModuleNotFoundErrorfor optional deps (openai/groq/ffmpeg/transformers).Test plan
vcon_processing.<chain>spans appear in SignOz withlink.*/storage.*spans nested under them, one trace per vCon.🤖 Generated with Claude Code