Add EXAONE 4.5 model support for Inference V2

[Bias92]

Add EXAONE 4.5 model support for Inference V2

summary

Add support for LG AI Research's EXAONE 4.5 language model in DeepSpeed Inference V2, by reusing the existing EXAONE 4.0 implementation.

EXAONE 4.5's text decoder is architecturally identical to EXAONE 4.0 (post-norm + QK-Norm + hybrid sliding/full attention) — text_config.architectures == ["Exaone4ForCausalLM"]. So this PR reuses Exaone4TransformerContainer and Exaone4InferenceModel directly and adds only the EXAONE 4.5 packaging differences.

changes

• New model implementation deepspeed/inference/v2/model_implementations/exaone4_5/:
  • container.py — non-transformer container for the multimodal checkpoint layout, where the LM weights are nested under model.language_model. (lm_head stays top-level).
  • model.py — Exaone4InferenceModel subclass; the policy passes the nested text_config, and activation_dtype is normalized for transformers v5 (dtype vs torch_dtype, str vs torch.dtype).
  • policy.py — extracts text_config, reuses Exaone4TransformerContainer under the model.language_model.layers prefix, and leaves the vision tower (model.visual.) and MTP head (mtp.) unmapped.
• checkpoint/huggingface_engine.py — derive max_seq_length from nested text_config.max_position_embeddings when the top-level (multimodal) config doesn't expose it.
• Register exaone4_5 in engine_factory.py and model_implementations/__init__.py.

why reuse 4.0

Verified against the real LGAI-EXAONE/EXAONE-4.5-33B checkpoint index (1064 tensors): all 64 decoder layers expose exactly the 11 parameters the 4.0 container maps, and every non-language-model tensor falls under model.visual. (342) or mtp. (15) — both declared unmapped. Nothing is left unaccounted for.

scope

Serves the base autoregressive text path of EXAONE 4.5 (e.g. LGAI-EXAONE/EXAONE-4.5-33B, bf16).

Out of scope / follow-ups:

• Vision tower (VLM, Exaone4_5ForConditionalGeneration)
• MTP self-speculative decoding
• FP8 / AWQ quantized variants (only fp16/bf16 supported, as in EXAONE 4.0)

As with EXAONE 4.0, rope_scaling (llama3) and per-layer sliding windows are not applied.

testing

Built the engine for LGAI-EXAONE/EXAONE-4.5-33B (bf16) on an A100 80GB (SXM4) via build_hf_engine — transformers 5.13.0, this branch (84e5b604). Both checkpoint shards load and every container initializes: parameter mapping and the model.visual. / mtp. unmapped-skip are both validated (is_initialized passes over all 64 layers).

[2026-07-05 18:31:23] [INFO] [engine_v2.py:84] Model built.
[2026-07-05 18:31:23] [INFO] [kv_cache.py:135] Allocating KV-cache 0 with shape: (64, 1, 64, 2, 8, 128) consisting of 1 blocks.
[A] engine built OK
[A]   num_layers           = 64       (expect 64)
[A]   model_dim            = 5120     (expect 5120)
[A]   n_heads / n_heads_kv = 40 / 8   (expect 40 / 8)
[A]   head_size            = 128      (expect 128)
[A]   vocab_size           = 153600   (expect 153600)
[A]   max_sequence_length  = 262144   (expect 262144)
[A] LOAD VALIDATION PASSED  (mapping / visual+mtp skip / is_initialized OK)
  gpu      : NVIDIA A100-SXM4-80GB
  branch   : add-exaone4_5-inference-v2 @ 84e5b604
  versions : torch 2.4.1+cu124 / transformers 5.13.0
  model    : LGAI-EXAONE/EXAONE-4.5-33B

Generation via DeepSpeed-MII is skipped here (mii not installed in the test env); load validation is the target of this PR. The KV cache is pinned tiny for the smoke test (`me

mory_config` ALLOCATE, 1 block) — this is a load/wiring validation, not a throughput benchmark.

Requirements

transformers >= 5.3.0 — EXAONE 4.5 landed in transformers 5.3 (huggingface/transformers#45471).

The commit was amended after the smoke test to add a DCO sign-off (d69ca55); its tree is identical to the tested 84e5b604.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d69ca559ff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Honor EXAONE 4.5 layer-specific attention

For the official EXAONE 4.5 text config, layer_types alternates sliding_attention layers with full_attention layers; the HF implementation applies RoPE only on sliding layers and passes a sliding-window mask only for those layers. By subclassing Exaone4InferenceModel unchanged here, DeepSpeed builds one RoPE-enabled dense-causal attention path for all 64 layers, so any generation through the full-attention layers (and prompts beyond the 4096-token sliding window) produces logits that do not match the checkpoint despite the engine loading successfully. The policy/model needs per-layer attention behavior instead of reusing the uniform EXAONE 4.0 path.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Apply scaled RoPE before accepting 262k contexts

For EXAONE 4.5 prompts past the original 8192-token RoPE range, this subclass still inherits Exaone4InferenceModel.positional_embedding_config, which only passes rope_theta and ignores text_config.rope_scaling={rope_type: 'llama3', factor: 16}. Because max_sequence_length is inherited from max_position_embeddings (262144), the engine will schedule those long contexts but rotate Q/K with unscaled frequencies, so generated logits diverge even when the checkpoint loads; either implement the scaling or cap/reject at the unscaled context length.

Useful? React with 👍 / 👎.