Skip to content

fix: resolve AWS Bedrock / cross-region Anthropic model IDs in model-info lookup#7930

Open
HumphreySun98 wants to merge 1 commit into
microsoft:mainfrom
HumphreySun98:fix/anthropic-bedrock-model-info
Open

fix: resolve AWS Bedrock / cross-region Anthropic model IDs in model-info lookup#7930
HumphreySun98 wants to merge 1 commit into
microsoft:mainfrom
HumphreySun98:fix/anthropic-bedrock-model-info

Conversation

@HumphreySun98

Copy link
Copy Markdown

Why are these changes needed?

anthropic._model_info.get_info() and get_token_limit() resolve a model's
capabilities / context window with an anchored str.startswith prefix match
against bare first-party model keys. AWS Bedrock model IDs carry a provider
namespace (anthropic.), an optional cross-region inference prefix (us.,
eu., apac., global.), and a -vN:M version suffix, e.g.
us.anthropic.claude-3-5-sonnet-20240620-v1:0. None of these match the bare
keys, so:

  • get_info() raises KeyError, which breaks AnthropicBedrockChatCompletionClient
    construction when model_info is not passed explicitly (re-raised as
    "model_info is required when model name is not recognized").
  • get_token_limit() silently returns the 100000 default instead of the real
    context window.

This PR normalizes the model id (strips the [<region>.]anthropic. prefix and
the trailing -vN:M suffix) before the lookup, so Bedrock IDs resolve to their
first-party model info. First-party IDs have neither part and are unaffected.

It also completes _MODEL_TOKEN_LIMITS with the Claude 4 models and -latest
aliases that were present in _MODEL_INFO but missing here, so get_token_limit()
no longer returns the default fallback for Claude 4 and the two tables stay in
sync (guarded by a new test).

Related issue number

Closes #7833

Checks

  • I've included any doc changes needed for https://microsoft.github.io/autogen/ (none required for this change).
  • I've added tests (if relevant) corresponding to the changes introduced in this PR.
  • I've made sure all auto checks have passed (ran ruff, mypy, and the relevant pytest locally).

…info lookup

`anthropic._model_info.get_info()` and `get_token_limit()` matched model
names with an anchored `str.startswith` prefix check. AWS Bedrock model IDs
carry a provider namespace (`anthropic.`), an optional cross-region inference
prefix (`us.`, `eu.`, `apac.`, `global.`), and a `-vN:M` version suffix, e.g.
`us.anthropic.claude-3-5-sonnet-20240620-v1:0`. These never matched, so
`get_info()` raised `KeyError` (breaking `AnthropicBedrockChatCompletionClient`
construction when `model_info` is not passed explicitly) and `get_token_limit()`
silently returned the 100000 default instead of the real context window.

Normalize the model id (strip the `[<region>.]anthropic.` prefix and the
`-vN:M` suffix) before the lookup so Bedrock IDs resolve to their first-party
model info. First-party IDs are unaffected.

Also complete `_MODEL_TOKEN_LIMITS` with the Claude 4 models and `-latest`
aliases that were present in `_MODEL_INFO` but missing here, so the two tables
stay in sync and `get_token_limit()` no longer returns the default fallback for
Claude 4 models.

Fixes microsoft#7833

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@HumphreySun98

Copy link
Copy Markdown
Author

@HumphreySun98 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

  • (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
  • (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree

@ErenAta16 ErenAta16 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independently re-derived _normalize_model and ran it against every Bedrock ID shape in the description plus the un-prefixed first-party case — all resolve correctly, and test_anthropic_model_client.py's new cases pass on the branch.

One thing worth flagging: #7940 is proposing the same fix for the same underlying bug on the same file, with a regex anchored to a fixed whitelist of region prefixes (us|eu|apac|global) instead of this PR's substring-split approach. I compared the two directly — for every region AWS documents today they produce identical output, but this PR's version degrades more gracefully if AWS adds a new region prefix later (it still strips the anthropic. marker regardless of what precedes it, where the anchored whitelist would silently fail to normalize an unlisted prefix and raise KeyError). This version also ships with dedicated tests, which #7940 does not. I'd lean toward this one and closing/superseding the other, but that's a maintainer call — leaving both my analysis and a note on #7940 so whoever triages it has the comparison.

@HumphreySun98

Copy link
Copy Markdown
Author

Thanks @ErenAta16 for the thorough independent verification and the comparison with #7940 — much appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic model-info lookup does not resolve AWS Bedrock / cross-region inference model IDs

2 participants