fix(testing): Fix PaliGemma 2 and PaddleOCR-VL test failures on main

[harshaljanjani]

What does this PR do?

The following failing tests were identified and fixed in this PR:

→ PaliGemma 2: The PaliGemma 1 test class contains additional_model_inputs = ["token_type_ids"] (needed since "token_type_ids": torch.zeros_like(input_ids) is present in inputs_dict), which is missing from the PaliGemma 2 test class, even though token_type_ids is assigned there as well; causes ValueError
→ PaddleOCR-VL: The base offload tests expect the model to split across GPU and CPU. Since the PR "Do not use accelerate hooks if the device_map has only 1 device", hf_device_map is only set inside this path. At the 70% split, the GPU budget (11.6MB) cannot fit embed_tokens (13.2MB), so get_balanced_memory places everything on the CPU and we end up with a single-device map. I think this also happens to be the same limitation handled in the Qwen2-VL tests (albeit with a vaguely listed reason); but PaddleOCR-VL was added later in 8c84144 without the corresponding skips. I found the limitation by instrumenting the code as follows:

cc: @Rocketknight1

CI Failures

Before the fix (feel free to cross-check; these errors are reproducible):

After the fix (feel free to cross-check):

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you fix any necessary existing tests?

[harshaljanjani]

cc: @Rocketknight1 Just a gentle ping :)

[Rocketknight1]

Yep, looks good to me! The 70% behaviour for device_map is interesting (cc @SunMarc) but the skips are fine for now

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: paddleocr_vl, paligemma2

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[harshaljanjani]

cc: @Rocketknight1 Apologies for the ping, but it looks like the PR was removed from the merge queue :)

[Rocketknight1]

I'll throw it in again!