ADR-206: Closed-vocabulary annealing actions with tiered escalation and epistemic ledger#403
Merged
Merged
Conversation
Two coupled fixes for Step 6 init failure ("Anthropic requires an
embedding provider") on first-run setup.
1. Reorder guided-init: configure the local embedding profile right
after admin user creation (new Step 4), before AI provider selection.
The wizard already collects the GPU/CPU choice at the top; map that
to a PyTorch device string (mac→mps, nvidia/amd/amd-host→cuda,
cpu→cpu) and pass it to `configure.py embedding --device`. The API
container picks up the active profile + device at startup. Renumber
subsequent steps (AI provider → 5, validate key → 6, model select →
7); Garage and start-app stay at 8 and 9.
2. SDK-direct catalog refresh. `models refresh` previously went through
get_provider(), which instantiates AnthropicProvider — whose __init__
eagerly constructs an OpenAIProvider as the embedding delegate. The
operator container has no loaded EmbeddingModelManager (only the API
container initializes one at startup), so get_embedding_provider()
returns None and the eager fallback fails for lack of an OpenAI key.
New _fetch_catalog_via_sdk() bypasses __init__ via __new__, sets
only the SDK client (or api_key for OpenRouter), and reuses the
existing fetch_model_catalog method. Mirrors the SDK-direct pattern
already used by _validate_provider_key.
Adds a --device flag to `configure.py embedding` so the wizard can
write the chosen device onto the activated profile in one call.
set_model_default fetched the provider/category for the target row and then unpacked the result with `provider, category = row`. When the caller's connection is configured with RealDictCursor (as the operator container's configure.py is — see operator/configure.py line 39), the row is a dict subclass and tuple unpacking silently yields the column *names* — "provider" and "category" — rather than the values. The clear-existing-default UPDATE then matched zero rows, and the set-new-default UPDATE collided with the still-set old default, violating idx_catalog_default on (provider, category). The API container's path didn't hit this because AGEClient.pool doesn't set a cursor_factory; only this operator-driven path tripped on it. Replace the SELECT + tuple-unpack with a subquery so the function is cursor-factory-agnostic. As a bonus the path is now idempotent: setting a model that's already the default no longer races with itself. Manifests as Step 7 of guided init: "Models command failed: duplicate key value violates unique constraint 'idx_catalog_default'".
ADR-200's binary promotion/demotion vocabulary cannot represent the merge-into-existing intent that diagnostics on proposals 35/36/37 showed the annealing agent actually needs — the LLM's own reasoning recognized the existing target ontology and recommended merging into it, but had no schema slot to express that, so the proposal failed identically across three cycles. This ADR redesigns the action menu as a closed 7-option set, introduces a tiered Sonnet → Opus → Human escalation cascade, makes the proposal queue a permanent epistemic ledger that Opus can mine when arbitrating, and exposes tuning controls so operators (and Opus, within safety rails) can self-regulate the cycle. Status: Draft — open for review on three unresolved design questions (outcome-quality metric basket, cost-budget safety rail, mid-cycle control-change UX). Related: ADR-200, #402
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Draft ADR extending ADR-200 to address three layered defects discovered during testing:
proposal_type ∈ {promotion, demotion}vocabulary is too narrow. Diagnostics on proposals 35/36/37 inkg_api.annealing_proposalsshow the LLM correctly identified a needed split-into-existing action ("carve a sub-cluster fromatlassian-api-bitbucket-dcinto existingatlassian-api-bitbucket-cloud"), but the schema had no slot for that intent — it fell back to "promote new ontology" with a colliding name and failed identically three times across three cycles.What this ADR proposes
Four phases — each independently shippable as a separate PR after this design is accepted:
Phase 1 — Closed action vocabulary. Replace binary promotion/demotion with a 7-action closed menu (
SPLIT_NEW,SPLIT_INTO_EXISTING,MERGE,DECOMPOSE_TO_PRIMORDIAL,RENAME,NO_ACTION,ESCALATE). Each action is structurally self-contained. LLM prompt grows to include the full ontology inventory. Pins the primordial pool as a permanent, undeletable reservoir — dissolution never destroys concepts.Phase 2 — Tiered escalation cascade. Sonnet classifies. Opus arbitrates as "evaluate the evaluator" (Sonnet's prompt and response are quoted as evidence in XML tags, not merged into Opus's task). Human is the final tier, with multi-turn dialogue support. The cascade is configured per platform via
escalation_chain— same scope as model provider / API key. Proposal reasoning lives in a newannealing_proposal_messagestable. GC becomes a load-bearing invariant: every proposal MUST reach a terminal state.Phase 3 — Control surface and self-regulation. Tuning knobs (
min_activity_for_cycle,min_ontology_age_epochs,golden_path_confidence,failure_cooldown_epochs, etc.) exposed via CLI/API/web. Opus can tune operational knobs via anADJUST_CONTROLaction; safety knobs (automation_level,escalation_chain,opus_confidence,phone_a_friend_cost_budget) remain admin-only rails.Phase 4 — Epistemic ledger. The proposal queue + reasoning chains become a permanent, mineable decision log. Terminal proposals are forever (no purging). Schema adds
signal_embedding,signal_payload,signal_kind,outcome_quality(async-set),superseded_by,graph_delta_summary. Opus's arbitration prompt becomes a RAG agent that injects retrieved similar past decisions as context. Calibration (confidence-vs-outcome correlation, human-vs-Opus agreement) becomes a closed loop feeding back into auto-tuning, not just observability.Open design questions — please weigh in
Three were surfaced during drafting and remain unresolved:
outcome_qualityas numeric, async-set, derived from post-execution graph metrics — but the metric basket (coherence drift, mass drift, cross-edge ratio change, …) is unspecified. This is load-bearing for Phase 4's calibration loop; needs calibration against real ledger data before pinning.phone_a_friend_cost_budgetas safety rail. Listed admin-only in the control table; the draft explicitly added it to the list of knobs Opus cannot self-tune. Confirm this is the intended behavior — a runaway Opus that can widen its own cost budget defeats the budget.Scope of this PR
This is a Draft PR for design review, not for merge. The ADR is
Status: Draft; once the open questions are resolved and reviewers concur on direction, status moves toAcceptedand Phase 1-4 implementation work branches off.Related
Test plan
This is a documentation-only PR; no code changes.
docs/scripts/adr lintis clean (verified locally)