Improve call site detection in capture mode and add it to hw decoder bench

[rostan-t]

Category:

New feature (non-breaking change which adds functionality)

Description:

Currently, the following fails to capture properly:

for _ in range(WARMUP_EPOCHS):
    run_epoch(reader)

for _ in range(NUM_EPOCHS):
    run_epoch(reader)

Operators were first captured in the warmup loop and the full call chain is used to identify a node so operators in the second loop fail to capture and fall back to normal execution.

This PR fixes this by trimming the call chain to only the required size. It also adds capture mode to L1_decoder_perf.

Additional information:

Affected modules and functionalities:

Dynamic capture mode.

Key points relevant for the review:

Does call chain trimming introduce silent regressions that I didn't think of?

Tests:

• Existing tests apply
• New tests added
  • Python tests
  • GTests
  • Benchmark
  • Other
• N/A

Checklist

Documentation

• Existing documentation applies
• Documentation updated
  • Docstring
  • Doxygen
  • RST
  • Jupyter
  • Other
• N/A

DALI team only

Requirements

• Implements new requirements
• Affects existing requirements
• N/A

REQ IDs: N/A

JIRA TASK: N/A

[dali-automaton]

CI MESSAGE: [61565034]: BUILD STARTED

[greptile-apps]

Greptile Summary

The PR compacts captured operator call chains to the minimum depth needed for call-site distinction and invariant proofs, allowing a captured graph to be reused across warmup and measured loops.

• Tracks source-analysis proof depth while classifying captured arguments.
• Compacts the capture trie before building the pipeline and stops lookup at compacted terminals.
• Adds warmup and nested proof-frame coverage.
• Adds optional dynamic capture mode to the hardware decoder benchmark and enables it in the decoder performance job.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
dali/python/nvidia/dali/experimental/dynamic/_capture.py	Adds proof-preserving call-trie compaction and early terminal lookup before captured-pipeline replay.
dali/python/nvidia/dali/experimental/dynamic/_source_analysis.py	Propagates the stack depth required to preserve cross-frame invariant proofs.
dali/test/python/experimental_mode/test_capture.py	Adds coverage for capture reuse across warmup loops and nested proof-frame call paths.
internal_tools/hw_decoder_bench.py	Adds an opt-in capture mode for the dynamic RN50 decoder benchmark and supplies capture-compatible invariants and RNG state.
qa/TL1_decoder_perf/test.sh	Enables dynamic capture for the L1 decoder performance benchmark.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Trace operator call] --> B[Build full call chain]
  B --> C[Classify arguments]
  C --> D[Record required proof depth]
  D --> E[Compact trie prefixes]
  E --> F[Build captured pipeline]
  F --> G[Later operator call]
  G --> H[Walk compacted trie]
  H --> I{Operator and arguments match?}
  I -->|Yes| J[Return captured pipeline output]
  I -->|No| K[Execute eagerly]

Loading

_{Reviews (3): Last reviewed commit: "Add ndd capture mode to decoder benchmar..." | Re-trigger Greptile}

[JanuszL]

Do you expect perf to match pipeline mode MIN_PERF?

[rostan-t]

When I measure on my workstation, even when tweaking parameters capture mode is consistently slower but the difference is under 3%.
This is not the same machine as used in CI, it's an RTX 6000 PRO but the CPU only has 16 cores.

I enabled L1 tests when triggering CI so we'll see.

[rostan-t]

Results from CI:

• Pipeline mode throughput: 21567.07 frames/sec
• Dynamic mode throughput: 17244.40 frames/sec
• Dynamic capture mode throughput: 21525.88 frames/sec

Looks like capture mode does match the performance of pipeline mode 🎉

[dali-automaton]

CI MESSAGE: [61575050]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [61565034]: BUILD PASSED

[dali-automaton]

CI MESSAGE: [61575050]: BUILD PASSED