Released pretrained world model appears largely insensitive to action conditioning — is this the Task-Centric config from the paper?

[changfeng2003]

Hi, and thanks for open-sourcing RISE — we've been using the released world model as an imagination environment for downstream RL.

We're seeing that the generated rollouts are largely insensitive to the action input, and before concluding anything we'd like to check our understanding with you.

Observed behavior

Using the released pretrained weights (and also our own task-level fine-tune on top of them), we fix the same initial frame + history and only vary the action sequence:

• real — the ground-truth actions
• zero — a no-op / static action
• reverse — the actions time-reversed

The three resulting videos are nearly identical. To rule out a trivial cause, we verified this on a high-motion segment (we picked the window with the largest np.diff(action), since early/static segments make all three actions numerically near-identical and would produce the same output regardless of the model).

Minimal repro:

1. Load released pretrained weights
2. Feed 4 real history frames + a high-motion action window
3. Swap real / zero / reverse actions, keep everything else fixed
4. Compare outputs → visually and numerically almost the same

What we've already checked

• The lightweight action path is wired in: the encoded action states are added to the text hidden states (action_encoder_control.py, encoder_hidden_states += action_states — [please verify line ~657 on your side]). So the conditioning is not silently dropped.
• The action signal itself is not degenerate: encoded act_states differ by ~[36]% between real and zero. The difference simply doesn't propagate to the output — it seems dominated by the video prior.
• Weight inspection: the released safetensors contain the act_vit action-encoder keys ([~77] keys) but no action_block* keys. Our reading of the paper (Sec. [XI-B]) is that the world model only uses the light-weight action encoder, and action_blocks belong to a different (policy-style) variant — so we believe the released weights are complete and consistent with the paper, not missing anything.

Our current understanding (please correct us)
From Table [V], the strong action controllability in the paper (flow EPE ≈ [0.54]) appears to rely on the Task-Centric Batching pretraining strategy, and without it EPE degrades to ≈ [0.68]. Our hypothesis is that the publicly released pretrained checkpoint may not include that Task-Centric gain, which would explain the weak controllability we observe — and if so, no amount of downstream fine-tuning on our end would recover it.

Questions

1. Is the released pretrained checkpoint the same configuration reported for the strong-controllability numbers in Table [V]?
2. Was Task-Centric Batching used when training the released weights?
3. If not, would it be possible to release a checkpoint trained with Task-Centric Batching (or share guidance on reproducing the paper-level action controllability)?

Environment

• GPU: [8× H20]
• torch [ver], diffusers 0.32.0, transformers 4.47.1
• weights: OpenDriveLab-org/RISE_Assets → pretrained

Thanks a lot for your time!