[TRTLLM-14880][feat] qualify Qwen3 dense for MX - #17142
Draft
chienchunhung wants to merge 2 commits into
Draft
Conversation
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
chienchunhung
force-pushed
the
codex/mx-qwen3-dense
branch
from
July 31, 2026 23:35
c65eebe to
24bd22c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@coderabbitai summary
Description
Add the second dense-model qualification profile for ModelExpress post-transform
weight reception. This PR reuses the exact runtime-constraint machinery from
#16974 and keeps the production change to a Qwen3-specific transform ABI plus an
exact
Qwen3ForCausalLMregistry row.The initial profile covers a Qwen3-8B-like dense configuration and explicitly
qualifies the Q/K-normalized attention lifecycle, fused QKV and gate-up layouts,
default fused QK-norm/RoPE path, and untied language-model head.
Dependency
This PR is stacked on #16974 and must merge after it. Because GitHub cannot use a
fork-only branch as the upstream PR base, the draft targets
mainand willtemporarily display the parent changes until #16974 merges.
#17029 is a parallel ModelExpress transport-delegation refactor. It does not add
model-family profiles, but it overlaps the loader integration and may require a
downstream sync if it lands before this stack.
Qualified Scope
Qwen3ForCausalLM/qwen3Unsupported runtime variants fail closed through the shared dense-Qwen support
matrix, including FP16, quantization, alternate attention backends, TP greater
than 2, PP/CP/EP expansion, LoRA, sparse attention, attention DP, multi-node
execution, tied embeddings, YaRN, unfused RoPE, and speculative decoding.
Test Coverage
deterministic output probes, with an unregistered-root negative control.
The focused PyTorch tests and real Qwen3 ModelExpress donor/receiver GPU E2E have
not run locally because this checkout has no PyTorch/Transformers runtime. They
remain required before the draft is marked ready.
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.