fix: resolve AWS Bedrock / cross-region Anthropic model IDs in model-info lookup#7930
fix: resolve AWS Bedrock / cross-region Anthropic model IDs in model-info lookup#7930HumphreySun98 wants to merge 1 commit into
Conversation
…info lookup `anthropic._model_info.get_info()` and `get_token_limit()` matched model names with an anchored `str.startswith` prefix check. AWS Bedrock model IDs carry a provider namespace (`anthropic.`), an optional cross-region inference prefix (`us.`, `eu.`, `apac.`, `global.`), and a `-vN:M` version suffix, e.g. `us.anthropic.claude-3-5-sonnet-20240620-v1:0`. These never matched, so `get_info()` raised `KeyError` (breaking `AnthropicBedrockChatCompletionClient` construction when `model_info` is not passed explicitly) and `get_token_limit()` silently returned the 100000 default instead of the real context window. Normalize the model id (strip the `[<region>.]anthropic.` prefix and the `-vN:M` suffix) before the lookup so Bedrock IDs resolve to their first-party model info. First-party IDs are unaffected. Also complete `_MODEL_TOKEN_LIMITS` with the Claude 4 models and `-latest` aliases that were present in `_MODEL_INFO` but missing here, so the two tables stay in sync and `get_token_limit()` no longer returns the default fallback for Claude 4 models. Fixes microsoft#7833 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@microsoft-github-policy-service agree |
ErenAta16
left a comment
There was a problem hiding this comment.
Independently re-derived _normalize_model and ran it against every Bedrock ID shape in the description plus the un-prefixed first-party case — all resolve correctly, and test_anthropic_model_client.py's new cases pass on the branch.
One thing worth flagging: #7940 is proposing the same fix for the same underlying bug on the same file, with a regex anchored to a fixed whitelist of region prefixes (us|eu|apac|global) instead of this PR's substring-split approach. I compared the two directly — for every region AWS documents today they produce identical output, but this PR's version degrades more gracefully if AWS adds a new region prefix later (it still strips the anthropic. marker regardless of what precedes it, where the anchored whitelist would silently fail to normalize an unlisted prefix and raise KeyError). This version also ships with dedicated tests, which #7940 does not. I'd lean toward this one and closing/superseding the other, but that's a maintainer call — leaving both my analysis and a note on #7940 so whoever triages it has the comparison.
|
Thanks @ErenAta16 for the thorough independent verification and the comparison with #7940 — much appreciated! |
Why are these changes needed?
anthropic._model_info.get_info()andget_token_limit()resolve a model'scapabilities / context window with an anchored
str.startswithprefix matchagainst bare first-party model keys. AWS Bedrock model IDs carry a provider
namespace (
anthropic.), an optional cross-region inference prefix (us.,eu.,apac.,global.), and a-vN:Mversion suffix, e.g.us.anthropic.claude-3-5-sonnet-20240620-v1:0. None of these match the barekeys, so:
get_info()raisesKeyError, which breaksAnthropicBedrockChatCompletionClientconstruction when
model_infois not passed explicitly (re-raised as"model_info is required when model name is not recognized").
get_token_limit()silently returns the100000default instead of the realcontext window.
This PR normalizes the model id (strips the
[<region>.]anthropic.prefix andthe trailing
-vN:Msuffix) before the lookup, so Bedrock IDs resolve to theirfirst-party model info. First-party IDs have neither part and are unaffected.
It also completes
_MODEL_TOKEN_LIMITSwith the Claude 4 models and-latestaliases that were present in
_MODEL_INFObut missing here, soget_token_limit()no longer returns the default fallback for Claude 4 and the two tables stay in
sync (guarded by a new test).
Related issue number
Closes #7833
Checks
ruff,mypy, and the relevantpytestlocally).