test: add tool_arg trailing-newline stream test (16_15)#36
Open
huahua-ctrl wants to merge 1 commit into
Open
Conversation
Add TestToolCallEdge::test_16_15_tool_arg_string_trailing_newline covering
a common real-agent scenario: a Write-style tool whose `content` string
argument must end with a literal `\n`. If the model or the service stack
silently rstrips trailing whitespace off tool arguments, `content` comes
back as `hello\nworld` (11 chars) instead of `hello\nworld\n` (12 chars),
and downstream file writers create malformed POSIX text files.
The case uses stream=True only. Rationale: stream is where the tail-token
loss (last chunk dropped on stop) surfaces — the same failure mode as the
existing SILENCE case (test_12_07_silence_literal_stream). Non-stream is
covered by the model behavior alone, which we already exercise indirectly
via the other 14_/16_ cases.
Cross-provider run against 10 direct providers (bodies match the pinned
M3.postman collection) shows a consistent split:
- PASS: M3-test / fireworks x2 / inferact x2 / magikcloud / modular
- FAIL: together GB300 / fp4 / B200 — all three return
{"content":"hello\nworld"} (11 chars, tail \n dropped)
This matches together's SILENCE case failure (tail `]` dropped) and gives
provider verifier two independent probes for the same class of "tail
character eaten on tool_call arguments stream" bug.
Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add TestToolCallEdge::test_16_15_tool_arg_string_trailing_newline covering a common real-agent scenario: a Write-style tool whose
contentstring argument must end with a literal\n. If the model or the service stack silently rstrips trailing whitespace off tool arguments,contentcomes back ashello\nworld(11 chars) instead ofhello\nworld\n(12 chars), and downstream file writers create malformed POSIX text files.The case uses stream=True only. Rationale: stream is where the tail-token loss (last chunk dropped on stop) surfaces — the same failure mode as the existing SILENCE case (test_12_07_silence_literal_stream). Non-stream is covered by the model behavior alone, which we already exercise indirectly via the other 14_/16_ cases.