Skip to content

perf(gb300): add AgentX aggregate TP4 points / 添加 GB300 AgentX 聚合式 TP4 数据点 #2676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ivanium wants to merge 1 commit into
base: main
Choose a base branch
from
+10 −1
Open

perf(gb300): add AgentX aggregate TP4 points / 添加 GB300 AgentX 聚合式 TP4 数据点 #2676

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7787,7 +7787,7 @@ dsv4-fp4-gb300-dynamo-vllm-agentic-mtp-agg:
ep: 1
dp-attn: false
- spec-decoding: mtp
conc-list: [8]
conc-list: [1, 2, 4, 6, 8]
prefill:
num-worker: 1
tp: 4
Expand Down
9 changes: 9 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6195,3 +6195,12 @@
- "Use AITER INT4 quick-reduce, ptpc_fp8 online quantization excluding embeddings, lm_head, gates, and experts, an FP8 KV cache, and three-token MTP with golden synthetic acceptance length 2.99 for benchmark runs."
- "Set max-num-seqs to twice the concurrency, select CUDA graph capture sizes by concurrency, and cap max-num-batched-tokens at 16384."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2576

- config-keys:
- dsv4-fp4-gb300-dynamo-vllm-agentic-mtp-agg
scenario-type:
- agentic-coding
description:
- "Expand the GB300 AgentX aggregate TP4 sweep to concurrency 1, 2, 4, 6, and 8."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2676
append-only: true
Loading