Add multilingual task loading to vLLM

[JoelNiklaus]

Problem

lighteval vllm could not run multilingual tasks such as lexam_mcq_4_idk:en from the built-in multilingual registry. Other backends already expose --load-tasks-multilingual, but the vLLM entrypoint did not, so users had to fall back to --custom-tasks or could not resolve the task at all.

Root Cause

main_vllm.py did not import the shared load_tasks_multilingual CLI argument and did not pass that value into PipelineParameters. As a result, the task registry was always initialized with load_multilingual=False for vLLM runs.

Solution

Add the shared load_tasks_multilingual option to the vLLM command and forward it to PipelineParameters, matching the behavior of the accelerate and sglang backends.

Testing

• Verified that Registry(tasks="lexam_mcq_4_idk:en|0", load_multilingual=True) resolves the LEXam task.
• Ran a one-sample vLLM smoke test for lexam_mcq_4_idk:en|0 with google/gemma-4-E2B-it after applying the cluster's cu129 vLLM upgrade recipe; the run completed end-to-end and saved results under results/gemma4-lexam-mcq4-idk-en-smoke/.

Made with Cursor

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.