Skip to content

[Power] Add optional B200 and B300 multinode telemetry / 添加可选 B200 和 B300 多节点功耗采集 - #2688

Draft
edwingao28 wants to merge 6 commits into
SemiAnalysisAI:mainfrom
edwingao28:feat/b200-b300-agentx-power
Draft

[Power] Add optional B200 and B300 multinode telemetry / 添加可选 B200 和 B300 多节点功耗采集#2688
edwingao28 wants to merge 6 commits into
SemiAnalysisAI:mainfrom
edwingao28:feat/b200-b300-agentx-power

Conversation

@edwingao28

Copy link
Copy Markdown
Collaborator

What changed

  • Port the H200-validated power launcher wiring to launch_b200-dgxc.sh, launch_b200-nscale-slurm.sh, and launch_b300-nv.sh.
  • Enable dcgm-power with required: false on 37 fixed-sequence recipes: 18 SGLang and 19 vLLM, including 15 dedicated-infrastructure topologies.
  • Route only power-enabled recipes through immutable producer e5c837f06a362dc888dfea2ee588e9f19c298270; preserve every existing non-power clone revision and keep Kimi-K3 and AgentX recipes unchanged.
  • Provision the DCGM exporter and record producer/exporter provenance in the power artifact.
  • Use fail-closed vLLM port ranges and pre-submit topology validation from the fork prerequisite.
  • Add dynamic contract and CI path coverage for all selected recipes.

Stack

  • This PR is stacked on InferenceX #2683 at 8073ca3fe3982146ca93ad6687b3c3aac8703f5c.
  • Producer prerequisite: edwingao28/srt-slurm #2 at e5c837f06a362dc888dfea2ee588e9f19c298270.
  • Review the Phase 3 commit after the PR-A commits in this draft; rebase onto main after PR-A merges.

Validation

  • CI-equivalent power and result-processing suite: 244 passed.
  • Official performance changelog validator passed against the PR-A base.
  • All 37 recipes load and pass complete topology/port preflight; the largest topology has 32 processes.
  • Three launchers pass bash -n; git diff --check passed.

Hardware status

No B200, B300, GB200, or GB300 job was dispatched. The current validation environment is limited to H100/H200, so all 37 recipe opt-ins remain required: false. Before rollout, an authorized environment must verify node port availability, exporter reachability, numa-bind vLLM paths, schema-v2 artifacts, role watts, whole-deployment joules, and dashboard points.

中文说明

变更内容

  • 将已经过 H200 验证的功耗 launcher 接线移植到 launch_b200-dgxc.shlaunch_b200-nscale-slurm.shlaunch_b300-nv.sh
  • 为 37 个固定序列 recipe 以 required: false 启用 dcgm-power:其中 18 个 SGLang、19 个 vLLM,并包含 15 个独立基础设施节点拓扑。
  • 仅让启用功耗的 recipe 使用不可变 producer e5c837f06a362dc888dfea2ee588e9f19c298270;保留全部现有非功耗 clone revision,且不修改 Kimi-K3 和 AgentX recipe。
  • 准备 DCGM exporter,并在功耗产物中记录 producer/exporter provenance。
  • 使用 fork 前置提供的 fail-closed vLLM 端口区间与提交前拓扑校验。
  • 为全部选中 recipe 增加动态 contract 与 CI path 覆盖。

依赖关系

  • 本 PR stacked 在 InferenceX #26838073ca3fe3982146ca93ad6687b3c3aac8703f5c 上。
  • producer 前置为 edwingao28/srt-slurm #2e5c837f06a362dc888dfea2ee588e9f19c298270
  • 本 Draft 中请在 PR-A commits 之后 review Phase 3 commit;PR-A 合入后再 rebase 到 main

验证

  • CI 同款功耗与结果处理测试:244 个通过。
  • 官方 performance changelog validator 基于 PR-A base 验证通过。
  • 37 个 recipe 全部可以加载并通过完整 topology/port preflight;最大拓扑包含 32 个 process。
  • 三条 launcher 均通过 bash -ngit diff --check 通过。

硬件状态

本 PR 未派发任何 B200、B300、GB200 或 GB300 作业。当前可用验证硬件仅限 H100/H200,因此 37 个 recipe 全部保持 required: false。正式 rollout 前,需要在获准环境中验证节点端口占用、exporter 可达性、numa-bind vLLM 路径、schema v2 产物、角色功耗、全部署焦耳口径及 dashboard 出点。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant