feat(evals): add OTEL tracer-provider module + deps (unwired)

[miguelg719]

why

The fan-out core for the opt-in OTEL transport: one NodeTracerProvider whose span processors route eval traces to Braintrust and LangSmith, each gated on its own key. Unwired — imported by nothing yet.

what changed

• framework/otel.ts (new): buildTracerProvider() (memoized; null unless EVAL_TRACE_TRANSPORT=otel; per-backend gated processors), getTracer(), null-safe shutdownTracing() (forceFlush+shutdown, 10s timeout)
• @opentelemetry/{api,sdk-trace-base,sdk-trace-node} deps
• tests/framework/otel.test.ts (new): native→null; both keys→2 processors; each alone→1

test plan

• Unit suite 362/362 (incl. the 4 new otel cases)
• Module unimported → no behavior change

Stack 2/5 · base: #2351

🤖 Generated with Claude Code

---

Summary by cubic

Adds an opt-in OpenTelemetry tracer provider for evals that can fan out spans to Braintrust and/or LangSmith, gated by their own keys. The module is added but not wired yet, and shutdown now tolerates forceFlush failures.

• New Features

  • New framework/otel.ts with buildTracerProvider(); returns null unless EVAL_TRACE_TRANSPORT=otel.
  • Span processors: Braintrust when BRAINTRUST_API_KEY is set (supports BRAINTRUST_OTEL_PARENT, BRAINTRUST_OTEL_URL, or an init braintrustParent override) and LangSmith when LANGSMITH_API_KEY is set and LANGSMITH_TRACING=true.
  • getTracer() and shutdownTracing() (10s timeout); shutdown proceeds even if forceFlush rejects; tests cover native mode, single/dual-backend setups, and shutdown behavior.

• Dependencies

  • Add @opentelemetry/api, @opentelemetry/exporter-trace-otlp-proto, @opentelemetry/sdk-trace-base, @opentelemetry/sdk-trace-node.

^{Written for commit 3055aa4. Summary will update on new commits.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: 3055aa4

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cubic-dev-ai]

All reported issues were addressed across 5 files

Confidence score: 5/5

• Safe to merge after the addressed issues were fixed.

Architecture diagram

sequenceDiagram
    participant Caller as Eval Runner (future)
    participant Builder as buildTracerProvider()
    participant Init as initializeTracerProvider()
    participant Env as Environment Variables
    participant BTExporter as OTLPTraceExporter (Braintrust)
    participant LSExporter as LangSmithOTLPTraceExporter
    participant Provider as NodeTracerProvider

    Note over Caller,Provider: NEW: OTEL Tracer Provider Initialization

    Caller->>Builder: buildTracerProvider()
    Builder->>Env: EVAL_TRACE_TRANSPORT
    alt Transport != "otel"
        Env-->>Builder: "native" or other
        Builder-->>Caller: null (no-op)
    else Transport == "otel"
        Env-->>Builder: "otel"
        Builder->>Init: initializeTracerProvider() (memoized)
        Init->>Env: BRAINTRUST_API_KEY?
        alt Key set
            Env-->>Init: key present
            Init->>Env: BRAINTRUST_OTEL_URL? else default url
            Init->>Env: BRAINTRUST_OTEL_PARENT? else project_name:stagehand-dev (or stagehand in CI)
            Init->>BTExporter: new OTLPTraceExporter(url, headers)
            BTExporter-->>Init: exporter instance
            Init->>Init: create BatchSpanProcessor(exporter)
        else Key missing
            Env-->>Init: no key
        end
        Init->>Env: LANGSMITH_API_KEY && LANGSMITH_TRACING=="true"?
        alt Both set
            Env-->>Init: keys set
            Init->>LSExporter: new LangSmithOTLPTraceExporter()
            LSExporter-->>Init: exporter instance
            Init->>Init: create BatchSpanProcessor(exporter)
        else Missing
            Env-->>Init: not set
        end
        alt spanProcessors.length > 0
            Init->>Provider: new NodeTracerProvider({ spanProcessors })
            Provider->>Provider: register() (global)
            Provider-->>Init: provider instance
            Init-->>Builder: provider
        else No processors
            Init-->>Builder: null
        end
        Builder-->>Caller: provider or null
    end

    Note over Caller,Provider: Shutdown flow

    Caller->>Builder: shutdownTracing()
    Builder->>Env: EVAL_TRACE_TRANSPORT == "otel"?
    alt Not otel or provider null
        Builder-->>Caller: return (no-op)
    else Valid provider
        Builder->>Provider: forceFlush()
        Provider-->>Builder: ok
        Builder->>Provider: shutdown()
        Provider-->>Builder: ok
        Builder->>Builder: reset provider to null
        Builder-->>Caller: done
    end

    Note over Caller,Provider: Tracer retrieval

    Caller->>Builder: getTracer()
    alt provider exists
        Provider-->>Builder: tracer from provider
    else No provider
        Builder-->>Caller: ProxyTracerProvider noop tracer
    end
    Builder-->>Caller: Tracer

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>