[AMD][MI35X] Sweep TP2/EP1 instead of TP2/EP2 on the Qwen3.5 MXFP4 MI355X AgentX arm - #2694
[AMD][MI35X] Sweep TP2/EP1 instead of TP2/EP2 on the Qwen3.5 MXFP4 MI355X AgentX arm#2694yichiche wants to merge 2 commits into
Conversation
…355X AgentX arm B200's qwen3.5-fp4-b200-sglang-agentic-mtp sweeps TP2/EP1, so the MI355X TP2/EP2 point was not comparable to any B200 column. Swap it for TP2/EP1 and extend the concurrency list to 32 to see where the 2-GPU arm tops out on GPU-resident KV. The TP4/EP1 arm and the benchmark script are untouched.
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM aside from the inline nit on the changelog PR-link. The config change itself (TP2/EP2 -> TP2/EP1, extended concurrency list) is simple, matches the stated motivation of aligning with the B200 arm, and is backed by passing matrix-logic tests and a local sweep-config dry run showing exactly the expected 19 points.
Extended reasoning...
Overview
This PR makes a single, narrow config change in configs/amd-master.yaml: it changes the 2-GPU arm of qwen3.5-fp4-mi355x-sglang-agentic-mtp from tp:2, ep:2 to tp:2, ep:1 and extends its concurrency list from [1,4,8,12,16,20] to [1,4,8,12,16,20,24,28,32], plus a corresponding append-only entry in perf-changelog.yaml. No benchmark scripts, images, or matrix-logic code are touched.
Security risks
None. This is a declarative YAML config change affecting benchmark sweep parameters (tensor/expert parallelism size and concurrency values). No user input handling, auth, or crypto code is touched.
Level of scrutiny
Low-to-moderate is appropriate: this is a config-only change to a benchmarking recipe, not production-critical serving code. The PR author documented local validation (pytest utils/matrix_logic -q passing, and a dry-run of generate_sweep_configs.py confirming the expected 19-point matrix with no leftover TP2/EP2 points).
Other factors
The one inline finding is a nit: the new perf-changelog.yaml entry points to PR #2693 instead of this PR (#2694), verified directly against the shipped commit content. Since perf-changelog.yaml is append-only, this is a permanent but non-functional attribution error that doesn't affect sweep generation or benchmark behavior. A separate candidate concern about the full-sweep-fail-fast label defeating exploratory-failure semantics was raised and ruled out by the bug-hunting system. Given the change is small, self-contained, matches an established pattern in the file (paralleling the B200 arm), and the only finding is a minor doc nit, this doesn't need to block on human re-review.
This comment was marked as outdated.
This comment was marked as outdated.
1 participant
Motivation
The 2-GPU point of
qwen3.5-fp4-mi355x-sglang-agentic-mtpis TP2/EP2, while its B200 counterpartqwen3.5-fp4-b200-sglang-agentic-mtpsweeps TP2/EP1. Those are different partitionings, so the 2-GPU columns of the two fleets are not comparable and the MI355X point does not answer the question the B200 point answers.The current TP2 arm also stops at concurrency 20, which is short of where the arm is likely to saturate, so the shape of the 2-GPU curve near its knee is unmeasured.
Modifications
In
configs/amd-master.yaml, change the 2-GPU arm ofqwen3.5-fp4-mi355x-sglang-agentic-mtpfromtp: 2, ep: 2totp: 2, ep: 1, and extend its concurrency list from1, 4, 8, 12, 16, 20to1, 4, 8, 12, 16, 20, 24, 28, 32.Nothing else changes. The TP4/EP1 arm keeps its concurrency list, the image is untouched, KV stays GPU-resident on both arms (
kv-offloading: none), andbenchmarks/single_node/agentic/qwen3.5_fp4_mi355x_sglang_mtp.shis not modified.The sweep grows from 16 to 19 points.
Accuracy Tests
No accuracy-affecting logic changes in this repo — the change is limited to the expert-parallel size and the concurrency list of one benchmark arm. The AgentX eval rows continue to run real target-model verification.
Benchmarking
Repo validation was run locally:
python -m pytest utils/matrix_logic/ -q→ 232 passed.python utils/matrix_logic/generate_sweep_configs.py full-sweep --config-files configs/amd-master.yaml --model-prefix qwen3.5 --precision fp4 --scenario-type agentic-coding→ 19 configs: TP2/EP1 at concurrency 1, 4, 8, 12, 16, 20, 24, 28, 32 and TP4/EP1 at concurrency 1, 4, 8, 12, 16, 20, 24, 28, 32, 40. No TP2/EP2 points remain.End-to-end MI355X AgentX numbers will come from the sweep triggered on this PR (
full-sweep-fail-fast). Concurrency 24 through 32 at TP2 is exploratory: with GPU-resident KV only, the top of that range may run out of device KV pool, and if it does, the failing points mark the 2-GPU ceiling rather than a regression.Related
#2693 makes a larger change to the same arm: it also adds a HiCache host-DRAM KV tier and kvdram arms. This PR is the KV-neutral subset — TP2/EP1 and the concurrency list only.