fix(providers/anthropic): report per-request usage at Finish - #50
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
馃挕 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: db67be9371
鈩癸笍 About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 馃憤.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: 鈩癸笍 About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 馃憤. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Anthropic requests on the Claude 5 family can run multiple internal iterations in one messages request. The final message_delta reports input, cache_read, and cache_creation tokens summed across all iterations, and the SDK accumulator merges those sums into acc.Usage. Consumers treat Finish usage as per-request context occupancy, so the summed values inflate context estimates and falsely trigger automatic chat compaction. Capture message_start usage and use its input and cache fields at Finish, falling back to the accumulator when message_start was never seen. Output tokens stay cumulative because all output was generated in this request.
1c2958f to
02fe208
Compare
|
Rebased onto the v0.40.0 sync (bb10946). The grpc/x/text bump commit was dropped: the sync's own dependency updates (grpc v1.83.0, x/text v0.40.0) supersede it, so the PR is now just the usage fix. Validated on the new base: red-green re-run (test fails on bb10946 without the provider change, passes with it), full Note for reviewers: the v0.40.0 sync does not fix this bug itself; the SDK fork's delta-merge (
|
|
@codex review |
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: 鈩癸笍 About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 馃憤. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
2 participants
Problem
Anthropic requests on the Claude 5 family can run multiple internal iterations in one messages request. The final
message_deltareportsinput_tokens,cache_read_input_tokens, andcache_creation_input_tokenssummed across all iterations, and the coder/anthropic-sdk-go fork'sAccumulatemerges those fields intoacc.Usagewhen present. The Finish stream part therefore carries cross-iteration sums.Consumers (coder/coder chatd) read Finish usage as per-request context occupancy (
input + cache_read + cache_creation), so a single multi-iteration response inflates the context estimate severalfold and falsely triggers automatic chat compaction. Verified production example: persistedcache_read = 287682where the real context was139956 + 7770(287682 = 139956 + (139956 + 7770)).Fix
Capture the
message_startusage during streaming. At Finish, use its input and cache fields, falling back to the accumulator whenmessage_startwas never seen. Output tokens stay cumulative from the accumulator because all output was generated in this request.Test
TestStream_FinishUsesMessageStartInputUsagestreams a two-iteration fixture (message_startwith iteration 1 usage, finalmessage_deltawith cross-iteration sums) and asserts Finish reports the per-request values. Red-green verified: without the provider change the test fails with the summed values.