Refresh B300 MiniMax M3 TRT AgentX with stream interval 20 - #2685
Refresh B300 MiniMax M3 TRT AgentX with stream interval 20#2685cquil11 wants to merge 2 commits into
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Claude finished @cquil11's task in 1m 7s —— View job PR Review
LGTM - no blocking issues found The change is a single-value edit ( |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — straightforward config refresh, no bugs found.
What was reviewed: the single stream_interval change (100→20) in the AgentX server YAML block, and the accompanying perf-changelog entry pointing to this PR. Checked that stream_interval is only referenced in this one config block and that no other B300 MiniMax M3 script logic (spec-decode force knob, KV offload settings) was touched. The ruled-out cross-SKU divergence with the B200 sibling was investigated by the bug hunting system and is not a functional issue.
Extended reasoning...
Overview
This PR changes a single TensorRT-LLM AgentX server config value (stream_interval: 100 → 20) in benchmarks/single_node/agentic/minimaxm3_fp4_b300_trt_mtp.sh, plus a corresponding append-only entry in perf-changelog.yaml documenting the refresh. No script logic, control flow, environment variables, or other config keys were touched.
Security risks
None. This is a benchmark tuning knob affecting how often the inference server flushes streamed token content to the client for latency measurement purposes. It has no bearing on auth, data handling, or execution paths.
Level of scrutiny
Low. This is a mechanical, single-value config tweak with a clear, well-explained rationale (avoiding collapsed streaming updates for short generations skewing inter-token latency metrics), consistent with a very similar prior change in this repo's history for a sibling B200 config. The perf-changelog entry follows the established append-only pattern.
Other factors
The bug hunting system found no bugs. One candidate issue (stream_interval divergence from the B200 sibling breaking cross-SKU comparability) was raised and refuted — different SKUs already have independently tuned configs, so this isn't a hard requirement for parity. Validation commands (bash -n, git diff --check) were run per the PR description.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32310415315 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32310415315 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32310415315 |
1 participant
Summary
Why
A stream interval of 100 can collapse short generations into a single content update. That makes client-observed inter-token latency and interactivity unrepresentative for those requests. A lower interval provides enough intermediate observations to measure the client experience more faithfully.
Performance changelog
Refresh with lower stream interval to collect correct client metrics
Validation
bash -n benchmarks/single_node/agentic/minimaxm3_fp4_b300_trt_mtp.shgit diff --check