fix: guard empty free_token_index in InferBatch._filter#1391
Conversation
For linear-attention mixed models (e.g. Qwen3.5), a request aborted before any kv is generated (cur_kv_len == 0) makes _linear_att_free_req return without appending anything. custom_cat then crashes with 'IndexError: list index out of range' on the empty list, killing the infer_loop thread. Skip custom_cat when the list is empty, matching the existing guard in pause_reqs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018e7Le4MGpdfA1LdPRShk3f
There was a problem hiding this comment.
Code Review
This pull request adds a check to handle empty free_token_index lists in infer_batch.py to prevent errors when calling custom_cat on linear attention hybrid models. The reviewer suggested simplifying the check using Python's idiomatic truthiness and removing the redundant else block.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| if len(free_token_index) != 0: | ||
| free_token_index = custom_cat(free_token_index) | ||
| else: | ||
| free_token_index = [] |
There was a problem hiding this comment.
The else block is redundant because free_token_index is already initialized as an empty list [] at the beginning of the method. Additionally, we can use Python's idiomatic truthiness check if free_token_index: instead of if len(free_token_index) != 0: to make the code cleaner and more readable.
| if len(free_token_index) != 0: | |
| free_token_index = custom_cat(free_token_index) | |
| else: | |
| free_token_index = [] | |
| if free_token_index: | |
| free_token_index = custom_cat(free_token_index) |
For linear-attention mixed models (e.g. Qwen3.5), a request aborted
before any kv is generated (cur_kv_len == 0) makes _linear_att_free_req
return without appending anything. custom_cat then crashes with
'IndexError: list index out of range' on the empty list, killing the
infer_loop thread. Skip custom_cat when the list is empty, matching the
existing guard in pause_reqs.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_018e7Le4MGpdfA1LdPRShk3f