Skip to content

Embed the coding-agent tracing daemon - #287

Open
Stephen Belanger (Qard) wants to merge 5 commits into
mainfrom
agent/bt-daemon-integration
Open

Embed the coding-agent tracing daemon#287
Stephen Belanger (Qard) wants to merge 5 commits into
mainfrom
agent/bt-daemon-integration

Conversation

@Qard

@Qard Stephen Belanger (Qard) commented Jul 29, 2026

Copy link
Copy Markdown

Dependencies

Depends on braintrustdata/braintrust-coding-agent-plugins#1, pinned at commit 3e019e47290318b1332c81311e64647b964c21b8.

That daemon PR in turn depends on braintrustdata/braintrust-sdk-rust#86, pinned at commit d33e806bf6ab9548d37355f6a5098a971ef150aa.

The plugin source cutover is intentionally deferred to stacked braintrustdata/braintrust-coding-agent-plugins#3, which must land only after this daemon is available in a released bt CLI.

Summary

This embeds the shared coding-agent tracing daemon in bt and exposes it as the bt agents command family. bt agents daemon runs the foreground service, while hook, status, and replay provide the integration and diagnostic entry points. The plugin hook shims can therefore use the installed bt executable for daemon lifecycle and transport without shipping another runtime.

bt agents setup codex and bt agents setup claude use the coding agents' own plugin managers to add the official Braintrust marketplaces and install the currently published tracing plugins. Setup also creates the shared tracing settings when needed, enables tracing, and accepts --project for routing. It performs a non-destructive merge: every other setting, including legacy authentication and backend fields needed by currently published plugins, is preserved unchanged. The published plugins can move authentication to the daemon after a bt release containing it is available.

The bt host resolves its normal profile, OAuth, keychain, or API-key credentials for each hook invocation and passes the resulting token, backend URLs, organization, and project to the credential-passive daemon. Hook errors remain fail-open so tracing cannot break a Codex or Claude Code turn.

The daemon is consumed from the exact Git commit of its migration PR rather than through a local path dependency. The integration is rebased on current bt main and preserves the newer top-level update command and command routing introduced there.

CLI regression tests verify the setup, daemon, hook, status, and replay command surface, the hook options, the Codex and Claude plugin-manager invocations, disabled Claude plugin recovery, default and explicit project selection, and preservation of existing metadata and legacy authentication settings. The superseded bt daemon path remains rejected.

Validation

  • cargo fmt --check
  • cargo check --locked
  • cargo test --locked
    • 725 unit tests and all CLI/eval/functions/setup integration suites passed.
  • cargo +1.92.0 clippy --locked --all-targets --all-features -- -D warnings
  • cargo test --locked --test cli agents_setup_
  • Real Claude Code setup in an isolated home:
    • added the official Braintrust marketplace
    • installed trace-claude-code@braintrust-claude-plugin
  • Real Codex marketplace listing succeeded in an isolated home. The subsequent marketplace clone was not completed locally because the current published repository is approximately 98 MB and the HTTPS fetch repeatedly stalled.
  • Manual isolated lifecycle smoke with the built bt binary:
    • started the embedded tracing service
    • forwarded Codex SessionStart and Stop events through the embedded hook
    • observed the session through the embedded status command
    • verified a redacted two-event journal
    • received /api/project/register, /version, and /logs3 at a local mock Braintrust backend
    • verified emitted root and turn rows, deterministic span IDs, project routing, and braintrust.plugin.codex span-origin metadata

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Latest downloadable build artifacts for this PR commit 3298cdf52986:

Available artifact names
  • artifacts-build-global
  • artifacts-build-local-x86_64-apple-darwin
  • artifacts-build-local-aarch64-pc-windows-msvc
  • artifacts-build-local-x86_64-pc-windows-msvc
  • artifacts-build-local-x86_64-unknown-linux-musl
  • artifacts-build-local-aarch64-apple-darwin
  • artifacts-build-local-x86_64-unknown-linux-gnu
  • artifacts-build-local-aarch64-unknown-linux-gnu
  • artifacts-plan-dist-manifest
  • cargo-dist-cache

Signed-off-by: Stephen Belanger <stephen.belanger@braintrustdata.com>
Signed-off-by: Stephen Belanger <stephen.belanger@braintrustdata.com>
@Qard
Stephen Belanger (Qard) force-pushed the agent/bt-daemon-integration branch from 39bbed4 to 6780c1d Compare July 30, 2026 13:55
Add bt agents setup codex and bt agents setup claude. The commands use each agent's plugin manager to install the currently published Braintrust tracing plugin without configuring the unreleased embedded daemon.

Signed-off-by: Stephen Belanger <stephen.belanger@braintrustdata.com>
Create the shared tracing settings file when missing and allow --project to select the trace project. Existing settings, including legacy authentication fields, are preserved unchanged so current published plugins continue to work.

Signed-off-by: Stephen Belanger <stephen.belanger@braintrustdata.com>
@Qard
Stephen Belanger (Qard) force-pushed the agent/bt-daemon-integration branch from 3fd4543 to 9f346b6 Compare July 30, 2026 15:14
Point bt at the daemon PR commit that restores src to main, keeping the existing published plugin runtimes independent from the daemon-capable CLI release.

Signed-off-by: Stephen Belanger <stephen.belanger@braintrustdata.com>
@Qard
Stephen Belanger (Qard) force-pushed the agent/bt-daemon-integration branch from 9f346b6 to 3298cdf Compare July 30, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant