Improve js-memory leak signal scoping and confidence#19
Conversation
|
Manual verification against the PR branch on iOS simulator looks good. Setup
Scenarios run1. Mixed history + temporary allocationPreloaded daemon history with unrelated samples, then ran a bounded temporary-allocation workflow. Commands: pnpm agent-cdp memory usage sample --label old-1
pnpm agent-cdp memory usage sample --label old-2 --gc
pnpm agent-cdp memory usage sample --label temp-baseline --gc
pnpm agent-cdp memory usage sample --label temp-action
pnpm agent-cdp memory usage sample --label temp-cleanup --gc
pnpm agent-cdp memory usage leak-signal
pnpm agent-cdp memory usage leak-signal --since jm_3Observed: This matches the intended behavior: full-history mode warns about mixed samples, while the bounded temporary-allocation window stays negative. 2. Retained allocation positive caseUsed the Commands: pnpm agent-cdp memory usage sample --label retained-baseline --gc
pnpm agent-cdp memory usage sample --label retained-action
pnpm agent-cdp memory usage sample --label retained-cleanup --gc
pnpm agent-cdp memory usage leak-signal --since jm_6
pnpm agent-cdp memory usage leak-signal --since jm_6 --verboseObserved compact output: Observed verbose output: This is the strongest confirmation for the PR: the positive result is driven by the post-GC retained-growth checkpoint, and the output explicitly communicates confidence, scope, quality notes, and caveat text. 3. Trend-only bounded run without a GC checkpointRan a bounded two-sample retained-growth window without any GC-assisted sample in the window. Commands: pnpm agent-cdp memory usage sample --label trend-baseline
pnpm agent-cdp memory usage sample --label trend-action
pnpm agent-cdp memory usage leak-signal --since jm_11
pnpm agent-cdp memory usage leak-signal --since jm_11 --verboseObserved: Verbose quality note correctly called out the missing GC checkpoint: This also matches the PR intent. 4. Too few samplesCommand: pnpm agent-cdp memory usage leak-signal --since jm_12Observed: 5. Invalid sample idCommand: pnpm agent-cdp memory usage leak-signal --since does-not-existObserved: ConclusionManual device testing supports the change set:
One minor observation: the mixed full-history temporary case reported |
…leak-signal # Conflicts: # packages/agent-cdp/src/__tests__/cli.test.ts
Summary
agent-cdpusers can now runmemory usage leak-signal --since SAMPLE_IDto evaluate one bounded interaction instead of accidentally scoring against older daemon samples.Context
The
js-memoryleak signal previously scored against the full stored sample history and could produce a strong-looking result from mixed workflows. This change keeps the existing sampling workflow, adds an opt-in bounded analysis window, and reworks the scoring/output so retained post-GC growth carries the most weight while trend-only evidence is explicitly downgraded.Backward-compatibility analysis:
memory usage sample,list,summary,diff,trend, andleak-signalworkflows continue to work.memory usage leak-signalkeeps the same command name and remains heuristic, but its default text output is now richer because it includes confidence/scope/quality notes.--since SAMPLE_IDflag is additive and does not change daemon lifecycle or stored sample format.Risks:
memory usage leak-signaltext output may need updates because the default formatter now prints confidence and quality-note text.--since.Manual testing:
agent-cdp memory usage sample --label baseline --gc, trigger the suspicious interaction, runagent-cdp memory usage sample --label action, then runagent-cdp memory usage sample --label cleanup --gc.agent-cdp memory usage leak-signaland verify the output calls out full-history scope or other low-confidence quality notes when applicable.agent-cdp memory usage leak-signal --since <baseline-sample-id>and verify the output reports bounded scope, includes confidence, and explains the post-GC retained-growth evidence for just that interaction window.Fixes #17