feat(webapp): run the dashboard agent through AWS Bedrock behind an env switch - #4609
feat(webapp): run the dashboard agent through AWS Bedrock behind an env switch#4609kathiekiwi wants to merge 19 commits into
Conversation
…switch DASHBOARD_AGENT_MODEL_PROVIDER=bedrock routes the agent's model calls through AWS Bedrock; the default stays direct Anthropic. Managed prompts keep canonical "anthropic:<model-id>" strings, resolved per provider in one place alongside the provider-shaped prompt-cache options.
Bedrock reports its cache write on the call's provider metadata and its cache read only on the call's usage, so the per-step cache telemetry reads whichever the active provider populates.
…ed fallback Confirm both Bedrock profiles against the SDK model-id union, throw on an unmapped id instead of guessing an unqualified profile, and add a structural shape test.
authenticateUserActor now applies the same membership floor as authenticateSession: a delegated token for a user who is not a member of the scoped org/project is denied, instead of being handed a usable (read:all-by-default) ability. Platform admins stay exempt.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (4)
📜 Recent review details⏰ Context from checks skipped due to timeout. (14)
WalkthroughThe dashboard agent now supports Anthropic and Bedrock model providers. Environment validation accepts provider-specific AWS configuration. Shared model resolution and cache-breakpoint helpers replace Anthropic-specific runtime logic. Cache telemetry reads provider metadata and usage fallbacks. Delegated user-actor authentication now validates token subjects and enforces organization and project membership, with integration test coverage. Mergeability Score: 🔵 Low · up to The PR adds an opt-in Bedrock provider while preserving the Anthropic default; the remaining merge-readiness risk is a test cleanup path in step-cache.test.ts that can remove a pre-existing provider setting and make provider selection order-dependent. This is a bounded test reliability issue requiring owner awareness, not evidence of a production behavior regression. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Only load the user and run the membership floor when organizationId or projectId is present, restoring pre-P5 behavior for unscoped tokens (no user query, no denial). The user lookup falls back to the primary before failing closed, so replica lag can't spuriously 401 a just-created member.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…er switch The sampled-turn eval task resolved its judge with Anthropic directly, so under the Bedrock switch with no Anthropic key every sampled eval failed. It now follows the same provider seam as the agent.
…versations On Bedrock the turn-wide prefix marker and the per-step marker were byte-identical, so the step-strip pass removed the prefix on short chats, leaving the history uncached every step. The step marker now carries an explicit ttl so the prefix survives, matching the Anthropic path.
Use a nonexistent userId for the unscoped case so success can only hold if the lookup is skipped.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…EFAULT_REGION The Bedrock SDK reads only AWS_REGION; pass region explicitly so AWS_DEFAULT_REGION (common in AWS deploys) works too. Credentials stay on the SDK's own chain.
…ing the webapp gate
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
The Bedrock step marker discriminated itself with a ttl inside cachePoint, which the SDK copies verbatim into the Converse request. Move the prefix/step discriminator to a non-serialized top-level providerOptions key so the wire cachePoint is a plain {type:"default"}. The key's value is an object because the AI SDK validates message providerOptions as records.
1 participant
What & why
The dashboard agent can now run its model calls through AWS Bedrock instead of the direct Anthropic API, chosen by a single env switch. It's off by default (
DASHBOARD_AGENT_MODEL_PROVIDERunset ⇒anthropic), so merging changes nothing at runtime — the Bedrock path is a dormant branch until an operator sets the switch and AWS config. The default Anthropic path is byte-for-byte unchanged.This also carries a related tenant-isolation hardening for the agent's delegated token (kept together deliberately — both land the agent on Bedrock for HIPAA readiness). Refs: TRI-13251, TRI-11032.
What's inside
Provider seam —
internal-packages/dashboard-agent/src/model-provider.ts: the registry now holds bothanthropicandbedrock;resolveDashboardAgentModel()maps the canonical"anthropic:<id>"strings the managed prompts carry to the active provider, and the cache-breakpoint helpers emit the active provider's shape — AnthropiccacheControlvs BedrockcachePoint. Managed prompt strings stay canonical, so stored prompts don't change meaning. Unmapped model ids throw rather than shipping a guaranteed-404 profile. All agent, watch, compaction and title callsites route through the resolver; thedashboardAgentModelKeylocals override (test mock injection) is preserved.Cache telemetry —
step-cache.ts: cache token usage is read from the active provider (Anthropic reports it on provider metadata; Bedrock reports the write on metadata and the read via standard usage), sogen_ai.usage.cache_*is populated on both. This also fixes a latent ordering bug where step attributes could null-overwrite the prompt-cache read count.Webapp callsites —
dashboardAgentHeadStart.server.tsand the head-start route resolve the model and the cache breakpoint through the same seam, so the warm-up prefix and the following turn share one provider. The head-start firing gate is provider-aware: on Bedrock it gates onAWS_REGIONand lets the SDK resolve credentials (IAM role / static keys / session token / bearer), so a role-based deploy still warms; on Anthropic it staysBoolean(ANTHROPIC_API_KEY).app/env.server.tsgains the optional AWS vars and validatesDASHBOARD_AGENT_MODEL_PROVIDER.ANTHROPIC_API_KEYis untouched and not required on a Bedrock deploy.Tenant-isolation hardening —
internal-packages/rbac/src/fallback.ts: for a scoped context, the OSSauthenticateUserActornow applies the same membership floor as the session path — a delegated user-actor token whose user is not a member of the scoped org/project is denied (403). Unscoped tokens keep their prior behavior (no tenant claim, no lookup). The user lookup falls back replica→primary so replication lag can't spuriously 401 a just-joined member. Members and admins are unaffected. Previously this invariant held only through per-route discipline; this makes it structural.Enabling Bedrock (later, ops)
DASHBOARD_AGENT_MODEL_PROVIDER=bedrockidentically in both the webapp and the agent task container — the webapp warms the cache prefix and the task reads it, so a split would silently miss the cache.AWS_REGIONand provide credentials the Bedrock SDK can resolve (IAM role preferred). For v1 this runs without an Anthropic API key. Note: with no Anthropic key set, rollback is "turn the agent off", not "unset the switch" (unsetting falls back to the Anthropic provider, which then has no key).Testing
Unit tests cover both provider paths: the provider switch and per-provider cache shapes, a structural regex asserting Bedrock ids are real inference profiles (not an echo of the table), the split-metadata cache telemetry, and real-Postgres RBAC tests — member allowed, scoped non-member denied (org-only and project-only), missing user → 401, admin non-member exempt, unscoped success.
typecheck --filter webappand the dashboard-agent + rbac suites pass.