Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions packages/ai/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ export const route = Route.make({
})
```

Route defaults are request-shaping defaults such as `headers`, `limits`, `generation`, `providerOptions`, and `http`. Endpoint host/query belongs on the route endpoint. Selected `LanguageModel` values carry only model id, provider id, and the configured route value. Model capability/catalog metadata lives outside this package; protocol support is enforced by request lowering and typed `AIError`s.
Route defaults are request-shaping defaults such as `headers`, `limits`, `generation`, `providerOptions`, and `http`. Endpoint host/query belongs on the route endpoint. Selected `LanguageModel` values carry model identity and the configured route; low-level callers may also attach model-specific defaults and compatibility metadata. Model capability/catalog metadata lives outside this package; protocol support is enforced by request lowering and typed `AIError`s.

The four-axis decomposition is the reason DeepSeek, TogetherAI, Cerebras, Baseten, Fireworks, and DeepInfra all reuse `OpenAIChat.protocol` verbatim — each provider deployment is a 5-15 line `Route.make(...)` call instead of a 300-400 line route clone. Bug fixes in one protocol propagate to every consumer of that protocol in a single commit.

Expand All @@ -88,7 +88,7 @@ For providers where the URL is derived from typed inputs (Azure resource name, B
Provider-facing APIs are configured facades over route values. Endpoint/auth/resource/API-version setup happens before model selection, and model selectors accept only a model or deployment id:

```ts
const openai = OpenAI.configure({ apiKey, baseURL })
const openai = OpenAI.configure({ apiKey, baseURL, store: false })
const model = openai.responses("gpt-4o-mini")

const azure = Azure.configure({ resourceName, apiKey, apiVersion: "v1" })
Expand All @@ -108,17 +108,22 @@ Keep provider facades small and explicit:
- Resolve `apiKey` → `Auth` with `AuthOptions.bearer(options, "<PROVIDER>_API_KEY")` (it honors an explicit `auth` override and falls back to `Auth.config(envVar)` so missing keys surface a typed `Authentication` error rather than a runtime crash).
- Use separate top-level facades for products with different required setup, such as `CloudflareAIGateway` and `CloudflareWorkersAI`.

Provider facades and model-derived `LLMRequest.providerOptions` are provider-specific, so expose typed native options flat at those boundaries. Provider package settings keep deployment configuration separate from their typed `providerOptions` field, except facades such as OpenAI whose settings are already unambiguous when flat. The selected `LanguageModel<Options>` carries request-option typing; the route decodes the flat runtime record. Keep provider metadata namespaced because replay may contain metadata from multiple layers.

`Provider.make(...)` remains available for simple static provider definitions, but new built-in providers should prefer plain configured facades unless a helper removes real duplication without adding runtime behavior.

### Provider Package Entrypoints

Catalog-selected native providers use package-like export paths from `@opencode-ai/ai`. They are internal entrypoints in one npm package, not separately published provider packages. Every entrypoint implements `ProviderPackage.Definition` and exposes `model(modelID, settings)`, where settings are serializable provider configuration plus common `headers`, `body`, and `limits` overlays.
Catalog-selected native providers use package-like export paths from `@opencode-ai/ai`. They are internal entrypoints in one npm package, not separately published provider packages. Every entrypoint implements `ProviderPackage.Definition` and exposes `model({ id, settings, credential, defaults })`. Core selects and refreshes the optional `key | oauth` credential; the provider package interprets it as route authentication. Serializable provider settings remain separate from common `headers`, `body`, and `limits` defaults.

```ts
import { model } from "@opencode-ai/ai/providers/openai/responses"

const selected = model("gpt-5", {
apiKey,
const selected = model({
id: "gpt-5",
settings: {},
credential: { type: "key", value: apiKey },
defaults: {},
})
```

Expand Down
68 changes: 56 additions & 12 deletions packages/ai/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -305,19 +305,26 @@ const gateway = CloudflareAIGateway.configure({
}).model("workers-ai/@cf/meta/llama-3.1-8b-instruct")
```

Included providers: OpenAI, Anthropic, Google (Gemini), Google Vertex Gemini and Anthropic, Amazon Bedrock, Azure OpenAI, Cloudflare AI Gateway, Cloudflare Workers AI, GitHub Copilot, OpenRouter, xAI, Z.ai, plus generic OpenAI-compatible Chat and Responses entrypoints and an Anthropic Messages-compatible entrypoint.
Included providers: OpenAI, Anthropic, Google (Gemini), Google Vertex Gemini and Anthropic, Amazon Bedrock, Azure OpenAI, Cloudflare AI Gateway, Cloudflare Workers AI, OpenRouter, xAI, Z.ai, plus generic OpenAI-compatible Chat and Responses entrypoints and an Anthropic Messages-compatible entrypoint. GitHub Copilot remains a Core-owned AI SDK integration rather than an AI-package provider.

### Package-like entrypoints

Native catalog integrations load provider behavior through package-like entrypoints. These are export paths from the same `@opencode-ai/ai` npm package, not independently published packages. Each entrypoint exports the same `model(modelID, settings)` contract, and `settings` contains serializable provider configuration plus common `headers`, `body`, and `limits` overlays.
Native catalog integrations load provider behavior through package-like entrypoints. These are export paths from the same `@opencode-ai/ai` npm package, not independently published packages. Each entrypoint exports the same `model({ id, settings, credential, defaults })` contract. Core selects and refreshes the optional `key | oauth` credential, while the provider package interprets it as route authentication. Serializable provider settings remain separate from common `headers`, `body`, and `limits` defaults.

```ts
import { model } from "@opencode-ai/ai/providers/openai/responses"

const selected = model("gpt-5", {
apiKey: process.env.OPENAI_API_KEY,
headers: { "x-application": "opencode" },
limits: { context: 200_000, output: 64_000 },
const apiKey = process.env.OPENAI_API_KEY
if (!apiKey) throw new Error("OPENAI_API_KEY is required")

const selected = model({
id: "gpt-5",
settings: {},
credential: { type: "key", value: apiKey },
defaults: {
headers: { "x-application": "opencode" },
limits: { context: 200_000, output: 64_000 },
},
})
```

Expand All @@ -341,30 +348,57 @@ Tuned Vertex Gemini deployments use model ids shaped like `endpoints/1234567890`
```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/gemini"

model("gemini-3.5-flash", { project: "my-project", location: "global" })
model({
id: "gemini-3.5-flash",
settings: { project: "my-project", location: "global" },
defaults: {},
})
```

```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/chat"

model("deepseek-ai/deepseek-v3.2-maas", { project: "my-project", location: "global" })
model({
id: "deepseek-ai/deepseek-v3.2-maas",
settings: { project: "my-project", location: "global" },
defaults: {},
})
```

```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/responses"

model("xai/grok-4.20-reasoning", { project: "my-project", location: "global" })
model({
id: "xai/grok-4.20-reasoning",
settings: { project: "my-project", location: "global" },
defaults: {},
})
```

```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/messages"

model("claude-sonnet-4-6", { project: "my-project", location: "global" })
model({
id: "claude-sonnet-4-6",
settings: { project: "my-project", location: "global" },
defaults: {},
})
```

Provider facades such as `OpenAI.configure(...).responses(...)` remain the direct application API. Package-like entrypoints are the self-similar loading contract used when a catalog selects behavior by export path.
Provider facades such as `OpenAI.configure(...).responses(...)` remain the direct application API. Package-like entrypoints are the self-similar loading contract used when a catalog selects behavior by export path. The entrypoints listed above implement that contract and are covered by `test/provider-package.test.ts`.

## How OpenCode uses this package

Other provider exports listed above remain direct facades until they explicitly implement the package-like contract. Exporting a provider facade does not implicitly make it a catalog-loadable provider package.
OpenCode does not call provider facades directly from the CLI or server. Core owns the integration:

1. `packages/core/src/model-resolver.ts` resolves catalog metadata and an active integration credential into a `LanguageModel`. Native package entrypoints expose `model({ id, settings, credential, defaults })`; catalog packages without a native mapping fall back through Core's AI SDK adapter.
2. `packages/core/src/session/model-request.ts` lowers Session state, instructions, tools, and plugin hooks into one canonical `LLMRequest`.
3. `packages/core/src/session/runner/llm.ts` calls the yielded `LLMClient.Service` once per physical attempt and persists provider-neutral `LLMEvent`s.
4. Core owns retries, continuation, compaction, permissions, durable tool execution, and Session history. None of that orchestration belongs in this package.

Title generation, compaction, standalone generation, and transient Session generation also build `LLMRequest`s and use the same `LLMClient.Service`. Core's `AISDK` adapter wraps remaining Vercel AI SDK models in executable routes so native and fallback providers present the same request and event model to callers.

This separation is intentional: `@opencode-ai/ai` owns one model call, provider protocols, and transport; Core owns the durable agent runtime.

## Provider options & HTTP overlays

Expand All @@ -377,6 +411,16 @@ Request options in order of stability:

Route/provider defaults are overridden by request-level values for each axis.

Provider-specific facades accept their own options directly because the provider is already known:

```ts
const model = OpenAI.configure({
apiKey,
store: false,
reasoningEffort: "high",
}).responses("gpt-5")
```

The selected model supplies the provider-specific option type, so per-request overrides stay flat while the canonical runtime request remains provider-neutral:

```ts
Expand Down
13 changes: 6 additions & 7 deletions packages/ai/example/tutorial.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,12 @@ import { OpenAI } from "@opencode-ai/ai/providers"
const apiKey = Config.redacted("OPENAI_API_KEY")

// 1. Pick a model. The provider helper records provider identity, protocol
// choice, capabilities, deployment options, authentication, and defaults.
// choice, deployment options, authentication, and defaults. Catalog capabilities
// remain application-owned and are not part of LanguageModel.
const model = OpenAI.configure({
apiKey,
generation: { maxTokens: 160 },
providerOptions: {
store: false,
},
store: false,
}).model("gpt-4o-mini")

// 2. Build a provider-neutral request. This is useful when reusing one request
Expand Down Expand Up @@ -74,8 +73,8 @@ const streamText = LLM.stream(request).pipe(
Stream.runDrain,
)

// 5. Tools are typed with Effect Schema. Provider turns remain explicit:
// advertise definitions on the request, stream one turn, dispatch local calls,
// 5. Tools are typed with Effect Schema. Model calls remain explicit:
// advertise definitions on the request, stream one call, dispatch local calls,
// then persist/build follow-up history in the enclosing product flow.
const tools = {
get_weather: Tool.make({
Expand All @@ -102,7 +101,7 @@ const streamWithTools = Effect.gen(function* () {
console.log("tool result", event.name, dispatched.result)

// A durable agent would persist these messages before starting another
// raw model turn. This tutorial keeps the boundary visible instead.
// model call. This tutorial keeps the boundary visible instead.
const followUp = LLMRequest.update(request, {
messages: [
...request.messages,
Expand Down
3 changes: 3 additions & 0 deletions packages/ai/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ export type {
LanguageModelOptions as ProviderLanguageModelOptions,
} from "./provider.js"
export type {
Credential as ProviderPackageCredential,
Defaults as ProviderPackageDefaults,
Definition as ProviderPackageDefinition,
ModelInput as ProviderPackageModelInput,
Settings as ProviderPackageSettings,
} from "./provider-package.js"
47 changes: 44 additions & 3 deletions packages/ai/src/provider-package.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,21 @@
import { Auth } from "./route/auth.js"
import type { AuthOverride, RequiredApiKeyAuth } from "./route/auth-options.js"
import type { LanguageModel, ProviderOptions } from "./schema/index.js"

export interface Settings extends Readonly<Record<string, unknown>> {
readonly baseURL?: string
export interface Settings {}

export type Credential =
| {
readonly type: "key"
readonly value: string
readonly configuration?: Readonly<Record<string, unknown>>
}
| {
readonly type: "oauth"
readonly accessToken: string
}

export interface Defaults {
readonly headers?: Readonly<Record<string, string>>
readonly body?: Readonly<Record<string, unknown>>
readonly limits?: {
Expand All @@ -11,11 +25,38 @@ export interface Settings extends Readonly<Record<string, unknown>> {
}
}

export interface ModelInput<ProviderSettings extends Settings = Settings> {
readonly id: string
readonly settings: ProviderSettings
readonly credential?: Credential
readonly defaults: Defaults
}

export const routeDefaults = (input: Defaults) => ({
headers: input.headers,
http: input.body === undefined ? undefined : { body: input.body },
limits: input.limits,
})

export const bearerCredentialValue = (input: Credential) => (input.type === "key" ? input.value : input.accessToken)

export const bearerAuthOption = (input: Credential): AuthOverride => ({
auth: Auth.bearer(bearerCredentialValue(input)),
})

export const apiKeyOrBearerAuthOption = (
input: Credential,
competingKeyHeader: string,
): RequiredApiKeyAuth | AuthOverride =>
input.type === "key"
? { apiKey: input.value }
: { auth: Auth.remove(competingKeyHeader).andThen(Auth.bearer(input.accessToken)) }

export interface Definition<
ProviderSettings extends Settings = Settings,
Options extends ProviderOptions = ProviderOptions,
> {
readonly model: (modelID: string, settings: ProviderSettings) => LanguageModel<Options>
readonly model: (input: ModelInput<ProviderSettings>) => LanguageModel<Options>
}

export * as ProviderPackage from "./provider-package.js"
38 changes: 18 additions & 20 deletions packages/ai/src/providers/amazon-bedrock-mantle.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { Auth } from "../route/auth.js"
import type { Route as RouteDef, RouteDefaultsInput } from "../route/client.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { OpenAIChat } from "../protocols/openai-chat.js"
import { OpenAIResponses } from "../protocols/openai-responses.js"
import { BedrockAuth, type Credentials } from "../protocols/utils/bedrock-auth.js"
Expand Down Expand Up @@ -79,29 +79,27 @@ export const configure = (input: Config = {}) => {

export const provider = configure()

const config = (settings: Settings): Config => {
if (settings.auth === "bearer" && settings.apiKey === undefined)
const config = (input: ProviderPackage.ModelInput<Settings>): Config => {
if (!input.credential && input.settings.auth === "bearer" && input.settings.apiKey === undefined)
throw new Error("Amazon Bedrock Mantle bearer auth requires apiKey")
if (settings.auth === "sigv4" && settings.apiKey !== undefined)
if (!input.credential && input.settings.auth === "sigv4" && input.settings.apiKey !== undefined)
throw new Error("Amazon Bedrock Mantle SigV4 auth does not accept apiKey")
return {
apiKey: settings.auth === "sigv4" ? undefined : settings.apiKey,
baseURL: settings.baseURL,
credentials: settings.credentials,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
region: settings.region,
...ProviderPackage.routeDefaults(input.defaults),
apiKey: input.credential
? ProviderPackage.bearerCredentialValue(input.credential)
: input.settings.auth === "sigv4"
? undefined
: input.settings.apiKey,
baseURL: input.settings.baseURL,
credentials: input.settings.credentials,
providerOptions: input.settings.providerOptions,
region: input.settings.region,
}
}

export const chatModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (
modelID,
settings,
) => configure(config(settings)).chat(modelID)
export const responsesModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (
modelID,
settings,
) => configure(config(settings)).responses(modelID)
export const chatModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (input) =>
configure(config(input)).chat(input.id)
export const responsesModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (input) =>
configure(config(input)).responses(input.id)
export const model = chatModel
28 changes: 15 additions & 13 deletions packages/ai/src/providers/amazon-bedrock.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type { RouteDefaultsInput } from "../route/client.js"
import { Auth } from "../route/auth.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { ProviderID, type ModelID } from "../schema/index.js"
import * as BedrockConverse from "../protocols/bedrock-converse.js"
import type { BedrockCredentials } from "../protocols/bedrock-converse.js"
Expand Down Expand Up @@ -50,19 +50,21 @@ export const configure = (input: Config = {}) => {
}

export const provider = configure()
export const model: ProviderPackage.Definition<Settings>["model"] = (modelID, settings) => {
if (settings.auth === "bearer" && settings.apiKey === undefined)
export const model: ProviderPackage.Definition<Settings>["model"] = (input) => {
if (!input.credential && input.settings.auth === "bearer" && input.settings.apiKey === undefined)
throw new Error("Amazon Bedrock bearer auth requires apiKey")
if (settings.auth === "sigv4" && settings.apiKey !== undefined)
if (!input.credential && input.settings.auth === "sigv4" && input.settings.apiKey !== undefined)
throw new Error("Amazon Bedrock SigV4 auth does not accept apiKey")
return configure({
apiKey: settings.auth === "sigv4" ? undefined : settings.apiKey,
baseURL: settings.baseURL,
credentials: settings.credentials,
generation: settings.topP === undefined ? undefined : { topP: settings.topP },
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
region: settings.region,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
apiKey: input.credential
? ProviderPackage.bearerCredentialValue(input.credential)
: input.settings.auth === "sigv4"
? undefined
: input.settings.apiKey,
baseURL: input.settings.baseURL,
credentials: input.settings.credentials,
generation: input.settings.topP === undefined ? undefined : { topP: input.settings.topP },
region: input.settings.region,
}).model(input.id)
}
Loading
Loading