Skip to content

[TRTLLM-14880][feat] qualify Qwen3 dense for MX - #17142

Draft
chienchunhung wants to merge 2 commits into
NVIDIA:mainfrom
chienchunhung:codex/mx-qwen3-dense
Draft

[TRTLLM-14880][feat] qualify Qwen3 dense for MX#17142
chienchunhung wants to merge 2 commits into
NVIDIA:mainfrom
chienchunhung:codex/mx-qwen3-dense

Conversation

@chienchunhung

Copy link
Copy Markdown
Collaborator

@coderabbitai summary

Description

Add the second dense-model qualification profile for ModelExpress post-transform
weight reception. This PR reuses the exact runtime-constraint machinery from
#16974 and keeps the production change to a Qwen3-specific transform ABI plus an
exact Qwen3ForCausalLM registry row.

The initial profile covers a Qwen3-8B-like dense configuration and explicitly
qualifies the Q/K-normalized attention lifecycle, fused QKV and gate-up layouts,
default fused QK-norm/RoPE path, and untied language-model head.

Dependency

This PR is stacked on #16974 and must merge after it. Because GitHub cannot use a
fork-only branch as the upstream PR base, the draft targets main and will
temporarily display the parent changes until #16974 merges.

#16458  MX post-transform qualification foundation [merged]
`-- #16974  Qwen2 dense model-family support [draft, CI running]
    `-- this PR  Qwen3 dense model-family support [draft]

#17029 is a parallel ModelExpress transport-delegation refactor. It does not add
model-family profiles, but it overlaps the loader integration and may require a
downstream sync if it lands before this stack.

Qualified Scope

  • Exact root/config identity: Qwen3ForCausalLM / qwen3
  • Target-model post-transform transfer only
  • Dense BF16 with unquantized weights and KV cache
  • TRTLLM attention with default fused QK-norm/RoPE
  • TP1 and TP2; PP1, CP1, and EP1
  • Untied embeddings and no speculative mode or separately loaded draft model

Unsupported runtime variants fail closed through the shared dense-Qwen support
matrix, including FP16, quantization, alternate attention backends, TP greater
than 2, PP/CP/EP expansion, LoRA, sparse attention, attention DP, multi-node
execution, tied embeddings, YaRN, unfused RoPE, and speculative decoding.

Test Coverage

  • Added full-load versus staged-load lifecycle equivalence for Qwen3 dense TP1.
  • Added TP2 rank-local equivalence for ranks 0 and 1.
  • Added Q/K norm, fused-layout, transform-guard, parameter/buffer, and
    deterministic output probes, with an unregistered-root negative control.
  • Extended every shared dense-Qwen runtime rejection case to Qwen3.
  • Extended SourceIdentity transform-ABI binding coverage.
  • All changed-file pre-commit hooks pass.
  • Python 3.12 compilation passes for all changed Python files.

The focused PyTorch tests and real Qwen3 ModelExpress donor/receiver GPU E2E have
not run locally because this checkout has no PyTorch/Transformers runtime. They
remain required before the draft is marked ready.

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

@chienchunhung chienchunhung changed the title [TRTLLM-14727][feat] qualify Qwen3 dense for MX [TRTLLM-14880][feat] qualify Qwen3 dense for MX Jul 31, 2026
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
@chienchunhung
chienchunhung force-pushed the codex/mx-qwen3-dense branch from c65eebe to 24bd22c Compare July 31, 2026 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant