Skip to content

fix(tokenizer): treat special-token strings as text to prevent batch crashes (#3110)#3111

Merged
LauraGPT merged 1 commit into
modelscope:mainfrom
14790897:fix/tokenizer-special-tokens-3110
Jul 7, 2026
Merged

fix(tokenizer): treat special-token strings as text to prevent batch crashes (#3110)#3111
LauraGPT merged 1 commit into
modelscope:mainfrom
14790897:fix/tokenizer-special-tokens-3110

fix(tokenizer): treat special-token strings as text to prevent batch …

1e42316
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs