fix: agent exits promptly after a short trace drains - #25
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
gpufl trace --uploadhung for ~60s at the very end of short traces (e.g. the tutorials) even though the upload UI already showed "complete." The launcher would eventually hard-kill the agent and print a misleading "late windows may need a post-hocgpufl upload" warning.Root cause
In exit-when-drained mode,
GpuflAgent.awaitDrainThenExitonly declared a session "drained" after it had observed the live.tmp/window marker through its 1s poll (sawActive). A short trace creates and finalizes (removes) that marker between two polls, sosawActivenever flipped — the drain loop spun forever. The launcher then waited out its full--agent-drain-mscap (60s) and hard-killed the agent.Fix
Gate the drain on a cumulative signal instead of the race-prone
.tmp/observation:TailerManager.hasStartedAnySession()— backed by thestartedSessionskeyset, which is added to on discovery and never cleared.started && activeTailers == 0 && !anyActiveSession, two consecutive clean polls.The window-model
LogTaileronly decrementsactiveTailersafter a window is fully sent, so there's no premature-exit risk. The agent now self-exits ~2–3s after the upload drains.Refactor (folded in)
Centralized the agent's magic-number sleeps into a
Delaysutility with an interrupt-awaresleep()helper, replacing ad-hocThread.sleep/catch blocks acrossGpuflAgent,LogTailer,SessionWatcher, andTailerManager.signalSessionCompletealso gains exponential backoff and slf4j logging.Testing
./gradlew buildgreen (unit tests + jacoco).gpufl trace --uploadnow finishes immediately after "agent finished uploading" — no 60s hang.