feat(bazel): capture & forward Bazel stderr (phase 1)#1197
Draft
gregmagolan wants to merge 1 commit into
Draft
Conversation
d02a892 to
3a19c27
Compare
Spawn Bazel with a runtime-owned stderr instead of Stdio::inherit() when a task opts in, so its output can be pre-processed before reaching the terminal. Phase 1 forwards verbatim; the pipeline seam, capture modes, ordering, flushing, and memory bounds are the foundation for later output cleanup (dedup + count), pattern-matched hooks, and hung-server detection. - stream/output.rs: OutputStream reader thread — \n/\r record splitting, per-read flush (keeps the curses progress UI live), LineProcessor pipeline seam (pass-through only), EIO-as-EOF, 1 MiB carry cap, last_activity atomic (stall-watchdog substrate). Single producer->consumer with a blocking write so back-pressure is automatic (deliberately not the unbounded Broadcaster). - capture.rs: conditional capture fd — plain pipe (non-TTY) or PTY (interactive, via nix::pty::openpty) with slave-drop / CLOEXEC / winsize discipline; pipe fallback off-Unix. - build.rs: OutputProcessor type + CaptureMode; `output` param on Build::spawn overrides the stderr Stdio and starts the OutputStream post-spawn; output_stream joined in wait() after the child is reaped. - mod.rs: `output` arg on ctx.bazel.build/.test; bazel.output.processor() constructor + type. - axl-types: Writable::to_boxed_write() for the forwarder thread. - bazel.axl: capture_output opt-in on BazelTrait (default off). - bazel_runner.axl: build/pass the processor; decouple --isatty from stdout (--isatty=1 for PTY, --isatty=0 --curses=no for the pipe path). Off by default: with no opt-in, stderr stays Stdio::inherit() and behavior is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3a19c27 to
4457f55
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Today every Bazel invocation in AXL tasks (
ctx.bazel.build/ctx.bazel.test, both viabazel_runner.axl) spawns the Bazel child withcmd.stderr(Stdio::inherit()), so its output reaches the terminal untouched and we can't pre-process it.This adds an opt-in capture → process → forward path for the child's stderr: the runtime captures it, runs each record through an extensible pipeline, and forwards the survivors to the real stderr. Phase 1 forwards verbatim — the goal here is to get capture mode, ordering, flushing/snappiness, memory bounds, and lifecycle correct. It's the foundation for follow-ups: output cleanup (dedup repeated lines + count), pattern-matched hooks, and hung/locked-up-server detection.
Off by default. With no opt-in, stderr stays
Stdio::inherit()and behavior is byte-for-byte unchanged.How it works
BazelTrait.capture_outputis set,bazel_runner.axlbuilds a processor and decouples--isattyfrom stdout to match the captured fd:--isatty=1); forward the master bytes near-verbatim.--curses=no --isatty=0, so Bazel emits clean newline-terminated lines (the tractable substrate for the deferred dedup/match work).stream/output.rs, modeled on the BES stream): splits records on\nand\r(so the\r-driven progress UI stays live), flushes once per read, runs aLineProcessorpipeline (pass-through in phase 1 — the seam where dedup/matchers plug in), and writes to the real stderr. Single producer→consumer with a blocking write, so back-pressure is automatic — deliberately not the unboundedBroadcasterused for structured BES events. Handles LinuxEIO-as-EOF, caps the in-flight line buffer, and maintains alast_activityatomic for the deferred stall watchdog.OutputStreamis joined inBuild::waitafter the child is reaped, so all forwarded stderr is flushed before the task prints its terminal summary. The PTY slave is dropped right after spawn so the master observes EOF (the classic PTY-hang pitfall).Deferred to follow-ups (room left, not wired): dedup+count,
output_matchhooks, and the kill/unhealthy decision — which per design will live in the existingbazel_attempt_endhook.Changes are visible to end-users: no
ctx.bazel.build/.test,bazel.output.processor, and theBazelTrait.capture_outputfield)Test plan
OutputStreamunit tests (record splitting, CR/LF, partial-line flush, carry cap, processor drop) and 2 real-subprocess round-trips through the full capture machinery for both pipe and PTY (these prove no hang at EOF).ASPECT_CAPTURE_TEST=1 aspect build //examples/deliverable:py_deliverable 2>err.log(via a temp config gate, reverted) — Bazel's stderr forwarded to the file, exit 0, no hang; confirmed--curses=no --isatty=0viaASPECT_DEBUG=1.script -q(real TTY) — confirmed--isatty=1, output forwarded through the PTY master, exit 0, no hang.