Skip to content

[AMD][MI35X] 0821 deepseek-v4 sglang agentic benchmark #2705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1am9trash wants to merge 3 commits into from
+11 −1
Closed

[AMD][MI35X] 0821 deepseek-v4 sglang agentic benchmark #2705

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
678bb8b
Update image for sglang mi35x v4 agentic workload
1am9trash Aug 21, 2026
5861476
Merge branch 'main' into dsv4-mi355-sgl-agentic-0821
billishyahao Aug 21, 2026
5af5b42
Merge branch 'main' into dsv4-mi355-sgl-agentic-0821
seungrokj Aug 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions benchmarks/single_node/agentic/dsv4_fp4_mi355x_sglang_mtp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ export SGLANG_DSV4_REASONING_EFFORT=high
export SGLANG_USE_ROCM700A=0
export SGLANG_HACK_FLASHMLA_BACKEND=unified_kv_triton
export AITER_BF16_FP8_MOE_BOUND=0
export SGLANG_OPT_USE_AITER_BATCHED_GEMM=true

# Unified radix tree: per-component (full-attn / SWA) cache management for
# hybrid-attention models, plus proactive release of out-of-window SWA KV
Expand Down
2 changes: 1 addition & 1 deletion configs/amd-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1727,7 +1727,7 @@ glm5.2-fp4-mi355x-atom-agentic-mtp:
# collective under long-context prefill, so no DEP arm ships until that path is
# validated. conc 16 appears on both arms to isolate the host KV tier's gain.
dsv4-fp4-mi355x-sglang-agentic-mtp:
image: lmsysorg/sglang-rocm:v0.5.17-rocm720-mi35x-20260813
image: lmsysorg/sglang-rocm:v0.5.17-rocm720-mi35x-20260821
model: deepseek-ai/DeepSeek-V4-Pro
model-prefix: dsv4
runner: cluster:mi355x-amds
Expand Down
9 changes: 9 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6298,6 +6298,15 @@
- "Increase prefill mem_fraction_static from 0.72 to 0.85 and reduce HICACHE_RATIO from 4 to 3."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2643

- config-keys:
- dsv4-fp4-mi355x-sglang-agentic-mtp
scenario-type:
- agentic-coding
description:
- "Bump image to lmsysorg/sglang-rocm:v0.5.17-rocm720-mi35x-20260821"
Comment on lines 6299 to +6306

@claude claude Bot Aug 21, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 AGENTS.md declares perf-changelog.yaml append-only and byte-sensitive ('Preserve all existing bytes and separator whitespace, and append only at the tail'), but this PR rewrites the prior trailing separator line (two trailing spaces after the PR #2643 entry) into a bare empty line before appending the new entry. Please restore the original ' ' (two-trailing-space) separator line and append the new entry after it, untouched.

Extended reasoning...

What happened: perf-changelog.yaml is explicitly declared in AGENTS.md (line 21) as append-only and byte-sensitive: 'Preserve all existing bytes and separator whitespace, and append only at the tail.' This PR's diff to that file is not a pure append — it modifies the very last line of the pre-existing file content before adding the new entry.

The exact diff hunk:

   pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2643
-  
+
 
+- config-keys:
+    - dsv4-fp4-mi355x-sglang-agentic-mtp
...
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2705
+

Step-by-step proof:

  1. git show HEAD~1:perf-changelog.yaml | tail -n 2 | cat -A on the pre-PR tree shows the file's last line is $ — i.e. a line containing exactly two trailing space characters, no other content.
  2. The PR diff shows a - line containing those same two spaces (- ) being removed, and replaced with a + line that is completely empty (+).
  3. That is a modification of an existing line's bytes (stripping two trailing spaces from a pre-existing separator line), not an append at the tail — it happens before the new - config-keys: ... entry is appended.
  4. Confirmed post-PR: git show HEAD:perf-changelog.yaml shows that line is now bare (no trailing spaces) where it previously carried two.

Why this matters: The repo instructions call this out under 'Non-negotiable benchmark invariants,' explicitly to protect against this exact class of edit — presumably because some tooling or diffing process downstream depends on byte-stable history for this file (e.g. line-count/byte-offset based diffing across changelog entries, or simply to keep git blame/diff noise-free per entry). Even though this specific alteration is whitespace-only and doesn't break YAML parsing, it is a real, literal violation of an explicit rule the repository asks reviewers to enforce, and it was introduced fresh by this PR (the byte was fine at HEAD~1).

How to fix: When appending the new changelog entry, avoid touching the previous entry's trailing separator line. Concretely, the append should read as pure addition after the existing bytes (including the two trailing spaces), for example using printf '...' >> perf-changelog.yaml or an editor mode that never rewrites already-committed lines, rather than a tool that reformats/strips trailing whitespace across the whole file (e.g. some YAML formatters or editors with 'trim trailing whitespace on save' enabled).

- "Enable SGLANG_OPT_USE_AITER_BATCHED_GEMM for mi355x sglang dsv4 agentic workload."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2705

- config-keys:
- dsv4-fp4-gb300-dynamo-trt-agentx
scenario-type:
Expand Down