Skip to content

Fix MRoPE dimension expansion in vLLM adapter for hybrid Qwen models#4177

Merged
copybara-service[bot] merged 1 commit into
mainfrom
xfgu-rl-mrope
Jun 16, 2026
Merged

Fix MRoPE dimension expansion in vLLM adapter for hybrid Qwen models#4177
copybara-service[bot] merged 1 commit into
mainfrom
xfgu-rl-mrope

Conversation

@xuefgu

@xuefgu xuefgu commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Description

When serving models that use MRoPE — such as Qwen3.5 and Qwen3-Omni (use_mrope=True) — vLLM passes 3D position indices of shape (3, num_tokens).

The vLLM adapter was expanding this at axis=1, producing a shape of (3, 1, num_tokens). When broadcasted against key projections during RoPE application, it incorrectly inflated the key sequence dimension (resulting in (batch, max_num_tokens, heads, head_dim)), ultimately leading to a shape validation failure in the RPA kernel: ValueError: Expected k.shape=(33554432, 1, 256) to be equal to v.shape=(2048, 1, 256).

The solution expand at axis=-1 to obtain the correct shape for both MRoPE and non-MRoPE cases.

Tests

Manual tests in E2E runs.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@codecov

codecov Bot commented Jun 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...t/integration/vllm/maxtext_vllm_adapter/adapter.py 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@copybara-service copybara-service Bot merged commit d15c1b0 into main Jun 16, 2026
52 of 53 checks passed
@copybara-service copybara-service Bot deleted the xfgu-rl-mrope branch June 16, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants