Conversation
… codec
Foundation of the new-schema-model migration (golem 1.5.0 -> 2.0.0 value model):
- Phase 0: vendor current golem main WIT deps (golem:core@2.0.0, golem:agent@2.0.0,
+golem:secrets/+golem:tool); update the agent-guest world (agent/host + include
-> 2.0.0, +secrets); regenerate golem-types bindings, keeping the old @1.5.0
core/agent .d.ts alongside the new @2.0.0 ones so the migration can proceed
module-by-module; add the new specifiers to rollup external.
- Phase 1: port the vendor-neutral schema-model layer from golem-ts-sdk
(recursive SchemaType/SchemaValue/SchemaGraph, t.*/v.* ctors, SchemaBuilder,
GraphEncoder, mergeGraphDefs, schema-value-tree carriers), adjusted for
effect-golem's NodeNext tsconfig (.js import extensions).
- Phase 2: rewrite WitCodec as the Effect-Schema -> new-model walker producing
{ graph: SchemaGraph, isUnit, codec: Codec<Type, SchemaValue> }, composed with
the user's schema so refinements/transforms still run.
schema-model + WitCodec compile clean; downstream consumers (agent/method
dispatch, config, durability, rpc, element/multimodal) + tests still reference the
old model and are migrated in later phases.
method.ts, agent.ts, guest.ts now run on the new model: - MethodCodec carries per-param WitCodecs + a unit|single output (no ElementCodec / DataSchema); compileParamCodecs/compileMethodSpec produce them. - invokeSchemaValue decodes the input record per-param via Schema.decodeEffect + schemaValueFromWit, runs the handler, encodes via Schema.encodeEffect + schemaValueToWit -> option<schema-value-tree>. Typed-error result<S,E> folding and the void-success empty-record stand-in are preserved. - agent.ts assembles the AgentType via one GraphEncoder per agent (merge graphs -> input-schema parameters + output-schema unit|single; schema = encoder.finish()); dispatch on schema-value-tree. - guest.ts boundary -> SchemaValueTree, invoke -> option, export object renamed to golemAgent200Guest (wasm-rquickjs convention for golem:agent/guest@2.0.0). - Multimodal/unstructured params rejected pending the Phase 5 redesign. NOTE: WIP — does not fully compile yet. The core dispatch path (method/agent/guest/ WitCodec/schema-model) is type-clean; the rich-feature modules still on the old model (Config, Client/RPC, durableFunction, Quota, Element/Multimodal/Unstructured) are migrated in Phase 4-5.
- DurabilityClient / durableFunction: ValueAndType -> typed-schema-value via typedSchemaValueToWit / typedSchemaValueFromWit (paired with the codec's graph). - Config / ConfigClient: get-config-value now takes a schema-graph and returns a schema-value-tree; config value-types carry a SchemaGraph and are flattened into the agent's shared pool by agent.ts (one GraphEncoder), so each config declaration resolves to a type-node-index. Typed-agent-config-value -> typed-schema-value. - agent.ts assembles config value-types into the same GraphEncoder pool. Core dispatch + durability + config now type-clean. Remaining (still old model): Client (RPC), Quota/QuotaClient (golem:quota WIT changed), Element/Multimodal/ Unstructured (element layer removed -> Phase 5 redesign). 31 -> 21 src errors.
- Client (RPC): client-side dispatch on schema-value-tree (encodeParams -> v.record -> schemaValueToWit; decodeMethodOutput from option<schema-value-tree>; compileParamCodecs; CompiledClient.constructorCodecs). - RpcClient / ConfigClient: host wrappers -> golem:agent/host@2.0.0 (invoke-and-await + get-config-value on schema-value-tree / schema-graph). - Quota / QuotaClient: golem:quota free-function API (newToken/reserve/split/merge); quota-token is now an opaque core capability handle (record codec removed; the capability-node walker support is deferred to Phase 5). - Element: element-value layer removed -> componentModelElement is inert pending the Phase 5 Multimodal/Unstructured redesign. - Delete dead witTree.ts. `npm run build` (tsc -p tsconfig.build.json) is clean and emits dist. Remaining: test/** (Phase 6) + Multimodal/Unstructured/quota-token Phase 5 redesign; then rebuild agent_guest.wasm and verify the counter.
generate-agent-template.mjs now swaps the wasm-rquickjs skeleton's upstream wit-bindgen for Golem's golem-outline-lift fork (the golem:core@2.0.0 schema model produces giant lift/lower wrappers upstream can't handle). The full agent-template crate compiles to wasm32-wasip2 and wasm/agent_guest.wasm is regenerated.
- AgentHostClient: golem:agent/host@1.5.0 -> @2.0.0 (the value import that broke
module init in the guest); parseAgentId now returns a typed-schema-value, so
agent.ts uses ctorDataValue.value (the schema-value-tree) on snapshot-load.
- snapshotEnvelope: golem:core/types@1.5.0 -> @2.0.0 (uuidToString/parseUuid).
- Add a guarded strictTextDecoder() helper (try fatal, fall back to lenient) and
use it in KeyValue/Webhook/Blobstore/snapshotEnvelope — the QuickJS guest is
compiled without ICU, where `new TextDecoder(_, {fatal:true})` throws at load.
- vitest: alias the @2.0.0 agent/core (and golem:secrets) specifiers to the mocks.
- Migrate the wit-codec test assertions to the new model (graph.root.body / SchemaValue);
all codec tests pass. Rename guest -> golemAgent200Guest in the agent/snapshot/http tests.
Counter example: golem build now extracts agent types AND module init succeeds on the
new binary (rebuilt from HEAD to match the new value model).
- guest export: revert golemAgent200Guest -> `guest` (effect-golem's wasm-rquickjs looks up the short interface name `guest.discoverAgentTypes`, unlike golem-ts-sdk's package-qualified convention); keep a golemAgent200Guest alias for callers/tests. - Config: a Schema.Redacted secret config leaf now declares its value type as secret<plaintext> (t.secret wrapping the inner type) — the agent registry rejects a bare plaintext type for secret-sourced config (AGENT_SECRET_INVALID_CONFIG_TYPE). Validated: golem build -> deploy [OK] -> Counter agent type registered -> invoke value=0, increment=1, value=1 (durable state persists across calls).
A secret config leaf declares its value type as secret<inner>, so getConfigValue
returns an opaque secret handle, not the plaintext. On read, reveal the handle
(capability-gated) against the inner-type graph to obtain the inner value tree,
then decode that with the inner codec and wrap in Redacted. Local leaves are
unchanged. Add minimal vitest mocks for golem:secrets/{reveal,types}.
Validated: Counter.keyTail (reads a Schema.Redacted apiKey, returns its last 4
chars) -> "1234". The Counter now exercises every method end-to-end (value/
increment/add/owner/caller/currentGreeting/keyTail) incl. durable state, local +
secret config, a number param, and the principal.
- host-features-agent: golem:core/types@1.5.0 -> @2.0.0 (uuidToString) — it was the
last value import of the old core types, which broke module init once the agent
was loaded alongside the others.
- main.ts / golem.yaml: restore all agents except QuotaTester (its quota-token API
is pending the Phase-5 capability-node migration).
Validated end-to-end against a local server (all 14 agent types register + deploy [OK]):
- Counter / SqliteCounter: durable state (+ node:sqlite snapshot)
- Caller: wasm-RPC to a remote Counter (bump -> 1)
- KvAgent / BlobAgent: wasi keyvalue + blobstore (put/write -> true)
- Lookup: result<S,E> typed errors (fetch -> { ok: 7 })
- plus config (local + revealed secret), number params, and principal on Counter.
Unit tests: 629/659 pass; the 30 failures are all the deferred Phase-5 features
(quota-token, multimodal, unstructured).
agent/method/snapshot/snapshot-sqlite/config/http/durable-function tests + the RpcFake helper. Inputs build via schemaValueToWit(v.record([...])); results are SchemaValueTree | undefined (unit -> undefined); AgentType assertions use the new input-schema parameters(named-field) / output-schema unit|single shapes; the secret config tests return v.secret(handle) and drive the golem:secrets/reveal mock. All migrated files pass individually; full suite 629/659 (remaining failures are the deferred quota/multimodal/unstructured Phase-5 features).
- Quota.ts: replace the stubbed Schema.declare QuotaToken with a codec marked by
witQuotaTokenAnnotationKey; the WitCodec walker maps it to the schema-model
quota-token capability node (graph root t.quotaToken({})). Encode lowers a host
QuotaToken -> v.quotaToken(GuestQuotaTokenHandle.fromRaw(...)); decode takes the
handle. Drop dead old-model exports (Uuid/EnvironmentId/Datetime/QuotaTokenRecord).
Operational API (acquire/reserve/commit/split/merge/withReservation) unchanged.
- WitTypes.ts: add the marker annotation key.
- test mock golem-quota-types: rewrite from the old class API (toRecord/fromRecord)
to the new free-function interface (newToken/reserve/split/merge/Reservation.commit),
preserving the event-recording + failure-injection hooks.
- quota.test: drop the 5 obsolete record-model schema cases, add two new-model cases
(quota-token node + v.quotaToken round-trip), keep all operational tests. Remove the
obsolete quota-token-property.test.ts and the wit-drift quota-record block.
quota.test: 17 passed. Build 0 errors.
- Unstructured: UnstructuredText/Binary now compile to t.variant([inline->text|binary, url->url]) with metadata.role = unstructured-text|binary; restrictions use the new languages[]/mimeTypes[] keys; value codecs map to v.variant(inline|url). ElementSpec is a thin carrier holding a pre-built WitCodec (preserves isElementSpec + the MethodParam/BindableKeys machinery). tryGetter routes thrown decode errors onto the schema-error channel. - Multimodal: multimodal(shape) builds one WitCodec rooted at list<variant> + role multimodal, walking each member (ElementSpec reuses its variant; plain Schema via toWitCodec). compile() returns a WitCodec. - method.ts compileParamCodecs: accept ElementSpec/Multimodal params (drop the Phase-5 rejection); multimodal-must-be-sole-param validation centralized here. - Element.ts trimmed to ElementValueKindError (dead ElementCodec/componentModelElement removed). - Rewrite unstructured/multimodal tests to the new model (graph.root variant/list + role, inline text/binary restrictions, v.variant/v.list round-trips); behaviors preserved. unstructured+multimodal: 11 passed. Build 0 errors. Full suite 647 passed; remaining 9 failures are pre-existing (client.test old-model + wit-tree.test for the deleted module).
The new-model quota-token is an opaque capability handle (no toRecord()/expectedUse).
- acquire: mint the token to prove acquisition, echo the known resourceName + expected.
- splitMerge: split + merge complete without error as the observable proof; report the
deterministic expected-use figures (split moves child out, merge folds it back).
The operational methods (withReservation/manualReserve*/exhaustAndReject) use the intact
Quota.* API unchanged. Re-wire into main.ts + golem.yaml.
Validated end-to-end (all 15 agents deploy [OK]): acquire 5 -> {resourceName, expectedUse:"5"};
withReservationOk 3/2 -> "committed:2"; splitMerge 8/3 -> {parent:"5", child:"3", merge:"8"}.
Mirror of the dispatch-test migration for the client side: inputs built via schemaValueToWit(v.record([...])); encoded form is a SchemaValue; constructor/method input assertions decode via schemaValueFromWit (.tag==="record", positional .fields); fake responder outputs are bare schema-value-trees (unit -> undefined, single -> schemaValueToWit(sv)) per Client.ts decodeMethodOutput; the void+typed-error wire-violation case now checks the missing-value path; custom-error payload uses typed-schema-value shape. client.test: 33 passed.
- Re-vendor src/internal/schema-model/{model,wit}.ts from golem-ts-sdk main (numeric
type nodes now carry option<numeric-restrictions>); re-add .js import extensions.
Update wit/deps/golem-core-v2 (+golem-tool) and regenerate the bindings; refresh the
curated golem-types core/tool d.ts.
- Adopt restrictions: WitTypes adds `restrict({min,max,unit})` + witNumericRestrictionsKey;
WitCodec numericNode lowers them to t.<kind>(restrictions) with the right NumericBound tag
(unsigned/signed/float-bits), canonicalizing -0.0 float bits.
Tests: 659 pass (656 + 3 numeric-restriction round-trip cases). Build 0 errors.
…d, snapshot→snapshotting) Aligns effect-golem's public defineAgent option keys with the canonical fluent SDK names, and sweeps the adjacent internal naming to fluent's "id field" / `Id` vocabulary: - constructorParams → id; snapshot → snapshotting (agent-definition keys) - canonicalConstructorParams → canonicalId; constructorParamNames → idFieldNames; string/nonStringBindableConstructorParams → string/nonStringBindableIdFields - generic C → Id (agent.ts / httpTypes.ts / Client.ts) - "constructor param(eter)" → "id field" in comments + error/branded messages Pure source-API rename — no wire/behavior change. The Snapshot module, the bound-instance AgentInstance.snapshot, and the shared MethodParams/MethodInput types are unchanged. Tests updated in lockstep; build + 659 tests green.
…ess→returns)
Aligns effect-golem's public method() option keys with the canonical fluent SDK
names, and sweeps the adjacent internal naming to fluent's `Input` generic:
- params → input; success → returns (method-definition keys)
- generic Params → Input (method.ts / Client.ts / Http.ts / httpTypes.ts)
- stringBindableParams → stringBindableInputs; nonStringBindableParams →
nonStringBindableInputs
Pure source-API rename — no wire/behavior change. Durability.wrap({success}),
SQL bind `params`, and successVoid/MethodParams/MethodInput are unchanged. Tests
updated in lockstep; build + 659 tests green.
…emas
effect's numeric pins were bare `Schema.Number.pipe(annotate)` — no integer/range
check (decode of 999 or 3.7 passed) — and `restrict({min,max})` was a generic
`Schema.annotate` applicable to any schema, never locally enforced. Both fluent
(validating marker) and the base/decorator SDK (checkIntRange) already reject
out-of-range at runtime; this brings effect in line.
- Integer pins (Uint8..Int32) -> Schema.Int.check(isBetween({minimum,maximum}))
via intPin(); bigint pins (Int64/Uint64) -> bigPin() with
isGreaterThanOrEqualToBigInt/isLessThanOrEqualToBigInt. Rejects non-integer /
out-of-range on Schema.decode/encode (the invocation boundary). Float32/64
unchanged.
- restrict(opts) is now typed <S extends Schema.Schema<number> |
Schema.Schema<bigint>>(self: S) => S (compile error on String/Boolean/record),
adds real bound checks, and re-carries the pin's witType tag onto the final
annotation layer (the codec's resolveAt reads only the last check, so appending
bound checks would otherwise drop the width). Bonus: preserves the static
number/bigint type (was erased to unknown).
Wire-safe: same emitted WIT type, just adds runtime rejection + compile-time
safety. Tests: decode/encode rejection cases in wit-codec-restrictions.test.ts +
new wit-numeric-restrict.test-d.ts (restrict is numeric-only). 663 vitest +
typecheck green.
Tier-A unification had renamed method({ success }) -> method({ returns }) to
match fluent. Reconsidered: an effect handler returns Effect<A, E, R>, so
`success`/`error` name its two channels and pair with the sibling `error:` key
(canonical Effect/Result vocabulary); `returns` reads oddly next to `error:`.
- method: returns -> success (field, option key, spec.success, JSDoc/prose).
- `input` stays unified with fluent (params -> input kept).
- fluent keeps `returns` (handler returns a value directly; no `error:` sibling,
typed errors fold into the return via s.result).
Wire-safe: pure source-API rename, no emitted schema/WIT change.
Same gap the SDK rename left: the in-repo integration-test app still used the old option keys (it's e2e, not part of `npm test`). Align it to the renamed API: constructorParams -> id, params -> input, snapshot -> snapshotting. `success` kept.
Add a `readOnly?: boolean | ReadOnlyOption` option to `method(...)` / `MethodSpec` (plus a `withReadOnly` combinator), reaching parity with the base/fluent SDKs. `true` maps to the base default `until-write` cache policy; the object form selects `no-cache` / `until-write` / `ttl` and per-principal caching. `resolveReadOnly` in agent.ts lowers it into the emitted `AgentMethod.readOnly` (`read-only-config`), replacing the hardcoded `undefined`.
Add `Principal.PrincipalSchema`, an Effect Schema whose WitCodec lowers to the WIT `golem:agent/common` `principal` variant (oidc/agent/golem-user/anonymous), letting a Principal travel as ordinary data in a method param/return (not just the Principal capability service). The schema is annotated with a new `witPrincipalAnnotationKey`; the WitCodec Declaration walker recognises it and emits the variant graph + a host-Principal round-trip value pair.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.