[Spec 1252] Prompt architecture: single-owner rule across the prompt surface - #1278
Closed
waleedkadous wants to merge 106 commits into
Closed
[Spec 1252] Prompt architecture: single-owner rule across the prompt surface#1278waleedkadous wants to merge 106 commits into
waleedkadous wants to merge 106 commits into
Conversation
Surface inventory measured directly rather than estimated. Two findings reshaped the spec: - The largest duplication axis is the codev/ <-> codev-skeleton/ shadow tree: 63 files / 45,183 words present in both, 19 already drifted. Tier 2 wins in the resolver, so this repo runs the stale copies. - The drift is live: codev/protocols/spir/builder-prompt.md is missing the Multi-PR Mechanics and Verify Phase sections the skeleton ships, so this project's own spawn prompt has no verify-phase instructions. Recommends sequencing B -> A -> C (parity test, then shadow-tree removal, then ownership map) with scar rules exempt throughout.
Gemini APPROVE / Codex REQUEST_CHANGES / Claude COMMENT. All nine issues accepted; two corrected factual errors in iteration 1. Claude's key finding: protocol-drift-audit.ts (#1210) already implements shadow-drift detection and codev doctor already reports it. Iteration 1's claim that no such test exists was wrong. The gap is that nothing fails the build -- M2 narrowed to wiring the existing audit into a CI gate. Count correction: 17 drifted files (16 protocols + 1 roles), not 19; the 19 conflated drifted with local-only. Three local-only entries must survive deletion (release/ + two maintain/templates/ files), guarded by new test T8. Codex's structural fixes: M1-M7 now required and independent of the open architect questions; M8-M10 (deletion) conditional on Q1. Added Appendix A (ownership-map schema), B (compatibility audit), C (tiering selector and fallback). Found while verifying: copyProtocols/copyRoles in scaffold.ts are never called by init/adopt/update. Adopters therefore have no shadow tree and ours is a historical artifact -- A1 corrected, M9 added to delete the vestigial functions. Production consumers route through the resolver, so deletion is lower-risk than iteration 1 judged.
All nine issues accepted, none disputed. Documents the two factual corrections (protocol-drift-audit #1210 already exists; 17 drifted files not 19) and the two builder-originated findings from verifying Codex's feasibility concern (vestigial copyProtocols/copyRoles; allowlist accretion risk).
Porch advanced to the spec-approval gate after the rebuttal, so the revisions are reviewer-informed but not reviewer-re-validated. Recorded so the architect isn't misled by the earlier 'pending' placeholder.
Skeleton wins for all 17 drifted files, reconciled file-by-file: missing-content divergences take the skeleton; anything resembling deliberate local content is escalated, never silently overwritten. Governs M3; independent of Q1. Adds an explicit 'Pending -- do not act' clause freezing Q1 (shadow-tree deletion, M8-M10), Q3 (scar registry contents, M5) and Q4 until the architect relays answers. Iteration-2 review deliberately held per architect countermand: it runs once against the fully-amended spec rather than against a moving target.
D1 (Q2): skeleton authoritative for the 17 drifted files. D2 (Q1): shadow-tree deletion APPROVED -- M8-M10 promoted from conditional to required, plus new M11 local-unique content audit. M11 covers content INSIDE drifted files, not just whole local-only files, and fixes sequencing M11 -> M3 -> M8 so D1 reconciliation cannot destroy local-unique content before deletion is reached. Ambiguous classifications default to escalate. D3 (Q3): eight scar rules ratified (six + shellper-kill + Tower-restart); canonical wordings materially compressed to one or two sentences, with verbatim-replication and delete/reword CI protection applying to the compressed form. D4 (Q4): tiering cut -- removed S2, C4, Appendix C and the schema tier field; recorded in Non-goals as a deliberate descope of the issue's proposed direction #2, deferred to its own future issue. Also updated: Desired State, sequencing, risk table (two new Critical and High rows), T6, new T12, N1 note (compression now contributes to the reduction target), and iteration-1 log entries annotated as superseded rather than rewritten.
Gemini APPROVE / Codex REQUEST_CHANGES / Claude APPROVE. All 6 points accepted, none disputed. Codex CX-1 (most valuable): ownership-map completeness was not testably defined -- M4 only validated entries already in the map, so a map listing 3 of 40 instruction classes would pass every test. Fixed with a declared inventory boundary, mechanical candidate extraction, mandatory mapped/scar/out-of-scope disposition, and T12 failing on anything undispositioned. Added unprompted: T12 is itself validated against a seeded line so it cannot pass vacuously on an empty candidate set. Codex CX-2: escalated local-unique files had no defined end state. Added terminal states TS1-TS4, made 'pending escalation' explicitly non-terminal, set completion at all 76 copies resolved with zero open escalations, and added a TS3 + follow-up-issue escape hatch so an unanswered architect question cannot stall the project. Gemini: M2 allowlist decay now enforced rather than expected. Claude: tests renumbered T11/T12/T13 with cross-refs fixed; T13 split into automated CI check plus one manual real spawn; Q7 promoted to required M1 coverage. Two risk rows added (vacuous map, stalled escalation).
Eight phases following the spec's B -> A -> C sequencing: drift gate + baseline, local-unique audit, reconcile, compat audit + shadow-tree removal, scar registry, ownership map, dedup + measure, governance sync + end-to-end. Ordering decision beyond the spec's stated constraints: scar compression (Phase 5) runs after shadow-tree removal (Phase 4) rather than before. The spec fixes M11 -> M3 -> M8 and 'M5 green before Approach C' but is silent here; compressing eight rules across ~36 files when half are deleted two phases later is wasted work and risks reconciling a compression edit against a skeleton that never received it. Phase 5 still precedes Phase 7 dedup, satisfying the actual constraint. Four of eight phases complete before any duplicated word is removed -- reflecting that drift, not token count, is the urgent problem and D2's requirement that nothing codev-specific be lost.
Gemini APPROVE / Codex REQUEST_CHANGES / Claude APPROVE. All 6 points accepted, none disputed. No change to phase structure, ordering, or scope -- all six were precision fixes to existing phases. Codex CX-1: M10 has two clauses (resolver equivalence AND byte-identical assembled prompt); Phase 4 implemented only the first, and the gap was disguised by Phase 8's T13 appearing to cover it. Step 4d now snapshots each protocol's spawn prompt pre-deletion and asserts byte-identity after, with the distinction from T13 documented. Codex CX-2: Phase 5 and Phase 8 both claimed the CLAUDE.md/AGENTS.md scar edits. Phase 5 now owns every scar-wording edit on every surface; Phase 8 parity-checks N3 only. The dangerous reading had M5 green while the two most-read surfaces were still un-rewritten, right before Phase 7 strips text. Codex CX-3: scaffold test scope corrected -- only copyRoles has cases. Claude: Phase 2 audit table specified as machine-parseable (T11 parses it; free-form prose would make T11 unimplementable and invite weakening the plan's most important safeguard); Phase 7 reference format given; .mjs -> .ts.
Closes a real gap: the issue's architect guidance made impact measurement mandatory, and M6 word counts + structural tests measure the proxy rather than the effect. M5/T6 protected against a scar rule being deleted or reworded; nothing detected 'rule present, compliance dropped'. SPEC: new required criterion M12 -- (a) behavioural baseline captured before any prompt-content change, (b) verify phase re-running the same script over N=10 post-merge projects with no-regression judgment and rollback trigger, (c) A/B eval considered and deferred with follow-up. New Appendix D specifies metrics, sample and triggers. New T14. Metric set grounded in what the repo actually stores. Two requested metrics could not be delivered and are recorded as such rather than faked: gate-rejection counts are NOT minable (no 'rejected' state exists across all 201 projects; requested_at is overwritten not appended), and consult tokens are prospective-only (rolling 30-day local DB, no historical data -- raw consult logs are gitignored). Minable set is B1-B5 over the 17 SPIR projects with non-empty history, plus 211 reviews and 139 threads for scar-violation mining. Stated plainly: the project both adds served content (M3 repair) and removes it (compression, dedup), so a null result could be two effects cancelling; rollback therefore targets specific commits (Phases 5/7 -- the trims) and never Phases 1-4 (the repairs). Under-powered windows are recorded as inconclusive, not success. PLAN: baseline capture added as Phase 1 step 1b, completing before any phase that alters served prompt content; verify phase defined with window, procedure, rollback table and honest power limits.
Gemini APPROVE / Codex REQUEST_CHANGES / Claude APPROVE on the D5 behavioural-measurement amendment. Both Codex findings accepted. B2 was unmeasurable as specified. 'Rounds to unanimous approve' never resolves: 0 of 48 terminal plan phases across the 17-project baseline end with 3x APPROVE -- porch advances on builder rebuttal, not consensus. Verified independently before changing. Redefined as review rounds per plan phase (max iteration per plan_phase). Computing the real baselines exposed a second issue: B2's observed range is 1-2 with mean 1.12, too little variance to detect a subtle regression. B2/B4 demoted to advisory; B1 named the load-bearing metric with a concrete soft threshold (>25% relative on the measured 51.9% baseline = above ~64.9%). Measured baselines now published in Appendix D rather than promised: B1 51.9% (n=160 verdicts), B2 mean 1.12 (n=49 phases), B4 mean 3.06 (n=18 projects). T14 contradicted itself: it demanded 'same commit => same B1-B5' while B5 derives from a rolling 30-day machine-local DB. Determinism scoped to the committed-artifact metrics B1-B4; B5 labelled advisory and non-deterministic and excluded from every rollback trigger. All three reviewers independently confirmed the data-availability claims (gate rejections unminable, consult data prospective-only, 17 SPIR sample).
…drift; capture word + behavioural baselines Delivers M2 (drift gate), M6 (word baseline), M12a (behavioural baseline). DRIFT GATE. shadow-drift-gate.test.ts calls the existing auditProtocolDrift() (#1210) rather than reimplementing detection -- the gap was never detection, it was that nothing failed the build. Allowlist seeded with exactly the 17 currently-drifted files, each justified PENDING_RECONCILE for Phase 3 per decision D1; starting populated is deliberate, since a gate that fails on the commit introducing it is a gate someone disables. Two guards keep it self-limiting: entries require a non-empty justification, and a stale-entry test fails once an allowlisted file stops drifting. A BITES test seeds a real divergence in a temp workspace and requires detection, so the gate cannot pass vacuously. WORD BASELINE. ALWAYS_ON = 24,614 words, matching the spec estimate. AGENTS.md and the hot-tier files are excluded from the sum: AGENTS.md is the byte-identical twin (one loads per session) and the hot files are already inlined in CLAUDE.md's count -- counting either twice would inflate the baseline and flatter the Phase 7 result. BEHAVIOURAL BASELINE. B1 51.88% REQUEST_CHANGES (n=160), B2 mean 1.12 (n=49 phases), B4 mean 3.06 (n=18 projects) -- reproducing the figures published in spec Appendix D. B3 yields 45 candidate hits across 350 files, mostly false positives on inspection, which is why the script emits excerpts for human adjudication rather than a bare count. Gate rejections and historical consult tokens are deliberately not implemented; Appendix D section 2 records why the data does not exist. DEVIATION. The plan placed the behavioural script at root scripts/; it needs js-yaml, which a root script cannot resolve in this pnpm workspace. Logic now lives at packages/codev/src/lib/ prompt-behavior-metrics.ts beside the sibling audit libraries, with a thin CLI runner in packages/codev/scripts/. Spec and plan updated to the real paths. Full suite: 193 files, 3772 tests, 0 failures.
…puts from artifacts; capture B5 snapshot Addresses both Codex iter-1 findings (Gemini/Claude approved). Spawn-prompt and phase-task word counts were hardcoded one-off measurements; a Phase 7 rerun would have reported pre-trim numbers for the two largest always-on components, corrupting the N1 delta. Both now derive from resolved artifacts (tier-2 codev/ first, then skeleton). Per-project variable content (issue body, task-JSON boilerplate) is deliberately excluded -- it is not trimmable surface. Reproducible baseline: ALWAYS_ON = 21,856 (the earlier 24,614 included the variable content); spec N1 note updated, word-baseline output now committed. B5 forward snapshot (30-day consult stats: 3,349 calls, $1,478.01) appended to the behavioural baseline artifact, capture-dated, marked non-deterministic, excluded from T14 determinism and all triggers.
…ng project from its own baseline Porch's iter-2 checks caught a live contamination: this project's own iter-1 review verdicts landed in its status.yaml and shifted B1's corpus 160 -> 163, and the project's thread file (which discusses scar rules at length) had contributed a B3 hit to the committed baseline. The baseline is defined as the PRE-PROJECT state (M12), so the measuring project's artifacts are excluded by default: SELF_PROJECT_DIR for B1/B2/B4, SELF_FILE_PREFIXES (spir-1252_*, 1252-*) for B3's prose scan. Baseline artifact regenerated clean (160 verdicts, 349 files, 44 hits); new test pins the exclusion. The verify phase inherits the same exclusion via the defaults.
…7 rows, 4 escalations One row per shadow copy, machine-parseable by contract (T11 parses it): 60 byte-identical -> TS1, 13 drifted-rot -> TS1, 4 local-unique -> pending with architect escalations sent via afx. Count correction: 77 shadow copies, not the spec's 76 -- the mechanical enumeration includes consult-types/integration-review.md, which the spec's hand count missed. Escalation 1: max_iterations 8 (local) vs 3 (skeleton) in spir+aspir protocol.json -- CMAP loop bound, no config override exists for the field. Escalation 2: cwd packages/codev in air+bugfix protocol.json -- genuine monorepo functionality; recommended migration to .codev/config.json porch.checks (Spec #550) then skeleton json. Rot evidence recorded per row rather than asserted: TICK confirmed retired (no protocol dir in either tree; one stale porch test ref), so all local TICK references are rot; obsolete underscore gate IDs (spec_approval) prove spir/protocol.md's retained porch section predates the skeleton restructure; architect.md's hardcoded area/ recipes are already owned by CLAUDE.md, so the skeleton's generalized version loses nothing. T11: 5 tests green + phase-gated completion guard (zero pending at M11 completion) + a process guard failing if any file vanishes while pending or TS3-retain.
…ase 4, not Phase 2; automatic completion guard Codex's iter-1 review caught the plan contradicting itself: Phase-2 criteria demanded zero pending rows while the spec's sequencing designs for open escalations through Phase 3 (M3 skips escalated files) with resolution required before Phase 4 deletion (M8 gated on M11). The bug was the plan text, not the pending rows: satisfying the criterion as written would have forced the TS3 escape hatch on all 4 escalations within the hour of sending them, bypassing the architect discussion D2 guarantees. The escape hatch is for escalations that cannot be resolved, not for skipping the wait. Plan criteria amended accordingly. T11 gains an automatic completion guard that needs no human to remember it: the moment any audited file is missing from codev/, zero pending rows may remain anywhere -- deletion itself arms the assertion. This answers Codex's deeper point (a manual skip-flag lets the phase pass while violating its own criteria) more durably than premature resolution would have. Also: spec count corrected in place (77 shadow copies, not 76 -- the hand count missed consult-types/integration-review.md) and the audit header arithmetic fixed per Claude (73 protocol files, not 74).
… post-1252 world; reschedule T13(b) to verify CLAUDE.md/AGENTS.md: ~15 prose references to deleted codev/protocols/ paths repointed at codev-skeleton/ (the single owner); the RELEASE protocol pointer annotated as codev-local by design; byte-identity maintained. arch.md: Repository Dual Nature rewritten -- codev/protocols/ holds local-only protocols exclusively; the shadow-tree deletion, its drift history, and the enforcing tests are told where a reader will look. New single-owner invariant recorded in Invariants & Constraints. lessons-learned.md gains the project's durable lesson: a detector that reports without failing is a detector nobody reads (with corollaries on frozen baselines, grep-classification, and served-vs-authored metrics). arch-critical caps verified (10 facts, 33 lines; C6 was retired there in Phase 5). Follow-up issues filed: #1276 (multi-model tiering, deferred by D4, now specifiable atop the ownership map) and #1277 (controlled A/B eval, deferred by M12c, with the observational ceiling documented). T13(b) rescheduled to the verify phase by architect ruling on a discovered constraint: afx spawn has no branch selector -- every spawn branches from main HEAD, so a pre-merge probe would assemble the PRE-change tree and verify nothing about this branch. Documented in the spec as a constraint (not a skipped test), added to the verify checklist beside the M12b comparison; the pre-merge proxy is the nine byte-compared prompt snapshots. Full suite: 3,744 passed, 0 failures.
… references in arch.md + CLAUDE.md Codex iter-1: accepted as a class. The Phase-8 sync rewrote the sections I knew about and missed arch.md deep references (Quick Start pointers, protocol section headers, roles Location, consultant resolution) plus one CLAUDE.md quick-reference line -- leaving arch.md internally inconsistent. Swept: every codev/protocols|roles reference repointed at the skeleton, annotated as the empty tier-2 slot, or retained where it correctly describes resolution doctrine. N3 held. Full suite: 3,744 passed, 0 failures.
…lects the post-1252 layout Claude iter-1 COMMENT: the Directory Structure diagram still drew the deleted protocol subdirectories. The tree now shows codev/protocols/ as local-only (release/) with the framework-resolution note. A diagram is a reference too.
…uidance with the scar rule Codex iter-2: the always-on arch facts said "never delete manually (use afx cleanup)" beside a scar rule forbidding "afx cleanup + respawn" -- tension, not contradiction (cleanup retires FINISHED builders; the scar rule protects LIVE ones), but a reader should not have to reconstruct the distinction. Both lines now state it explicitly. Hot caps verified: 10 facts, 33 lines. Full suite: 3,744 passed, 0 failures.
…ol/templates; glossary Skeleton entry Codex iter-4, both accepted. The Important Notes repointing had fixed the dangling path while preserving the fetch-by-path instruction -- the 1011 bug class this project polices. Rewritten to delivery language (protocol text is inlined at spawn; templates arrive in phase tasks; skeleton path kept as orientation reference). Glossary Skeleton entry now describes source-tree + build-time embed + tier-4 runtime fallback instead of the stale copied-on-init story. Full suite: 3,744 passed, 0 failures.
Codex PR review: both artifacts had passed their human gates but still read as drafts. Standard approved/validated frontmatter added, status lines name their gates. (The commit-format point is rebutted: the non-matching commits are porch-generated chore(porch) state commits, the established convention across main's history.)
…apter PR #1278 will not merge (charter-level re-scope at the pr gate): a successor shrinks the surface radically before enforcement is rebuilt fit-for-purpose. Review now tells the true story: what the harvest PR lands (17-file reconciliation + knowledge artifacts), what stays on this reference branch, and that the scar-registry concept plus the eight ratified wordings move into the successor charter.
waleedkadous
added a commit
that referenced
this pull request
Jul 28, 2026
The harvest of Spec 1252 after the pr-gate decision not to merge the full machinery (PR #1278, closed unmerged; charter-level re-scope -- a successor shrinks the surface radically before enforcement is rebuilt fit-for-purpose). RECONCILIATION (the live bug): all 17 drifted codev/ shadow copies synced to the skeleton content they had silently fallen behind. Served SPIR/ASPIR builder prompts regain their Multi-PR Mechanics and Verify Phase sections and the 'Entering verify phase.' notification -- until now, adopters received verify instructions and this repo's builders did not. Architect rulings applied: E1 max_iterations: 3 for spir/aspir (decided on the measured evidence that review rounds run 1-2, mean 1.12); E2 the air/bugfix check cwds live in .codev/config.json porch.checks (Spec #550 mechanism), so the forked protocol.json files sync to the skeleton. Spec-746 pure-addition baselines re-derived for the two synced builder-prompts (frozen-file baselines break on any legitimate edit). KNOWLEDGE ARTIFACTS: the 77-row shadow-tree audit with architect rulings and a harvest note; word-count baseline and after-measurement (the honest -7.0% N1 record with its decomposition); the behavioural baseline (B1 51.88% REQUEST_CHANGES over 160 verdicts, self-excluded, B5 snapshot); the measurement tooling (surface script, behaviour metrics library + CLI + determinism test); the project review telling the true story including the non-merge decision; and the builder thread. The scar-registry concept and the eight ratified rule wordings move into the successor charter -- deferred, not discarded. Full suite from a normal checkout path: green (the three worktree-write-guard failures observed in the tmp-located build worktree are the documented /private/tmp-allowlist platform artifact; the suite is untouched by this change).
Contributor
Author
|
Closed unmerged by architect decision at the pr gate — harvested, not discarded. Waleed's charter-level call: the structural machinery this PR carries (shadow-tree deletion, scar registry + CI enforcement, ownership map + completeness machinery, dedup partials, CLAUDE.md restructure) is not worth carrying for this surface, because a successor project will shrink the prompt surface radically first (blog-principles judgment rewrite, >50% reduction target, frontier-models-only assumption) — and enforcement gets rebuilt fit-for-purpose around the post-shrink surface. This is a risk/reward re-scope, not a quality verdict: every phase closed with unanimous CMAP approval, and the audit + measurement work here directly shaped the successor's charter. The harvest → PR #1281: the 17-file drift reconciliation (the live bug — served SPIR/ASPIR prompts regain their Verify Phase / Multi-PR sections; rulings E1 Not dying, moving: the scar-registry concept and the eight D3-ratified rule wordings transfer into the successor charter, along with the ownership-map design and this project's baselines ( This branch ( |
This was referenced Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements Spec 1252 (#1252): every instruction on the ~150k-word prompt surface now has exactly one owning surface, enforced by CI — with the eight ratified scar rules as the deliberate verbatim-replicated exception.
The headline finding that reshaped the spec: a 77-file shadow tree under
codev/protocols|roles|consult-typesoutranked the installed skeleton in the resolver and had drifted in 17 files — including the served SPIR builder prompt, which had silently lost its entireVerify Phasesection. The project filed to fix prompt drift was itself spawned with a drifted prompt, and the existing detector (#1210) had been reporting it tocodev doctor, unread, for months.What changed (8 phases, all unanimous at final review)
shadow-drift-gate.test.tswires the existingauditProtocolDrift()into a build-breaking check (allowlist now empty); word + behavioural baselines captured pre-changemax_iterations: 3; E2: cwd →.codev/config.jsonporch.checks)Multi-PR Mechanics+Verify PhasecopyProtocols/copyRolesremoved;skeleton-embed-syncbyte-locks source ↔ embeddedscar-rules.yaml), enforced line-exactly everywhere they apply; delete/reword fails CIprompt-ownership.yaml) with a completeness contract: every normative line needs a disposition; cross-surface duplication cannot hide behind the catch-all; the companion doc is parity-checkedMeasurement, honestly
N1: 21,856 → 20,324 served always-on words = −7.0% (target was ≥20%, explicitly a target not a gate). The repair added words the prompts were supposed to have; the remaining surface is single-owned prose whose largest block is protocol semantics (a spec Non-goal). Full decomposition in
codev/resources/1252-word-after-phase7.md.M12 behavioural verify (post-merge): baseline committed (
1252-behavior-baseline.md, B1 = 51.88% REQUEST_CHANGES rate over 17 SPIR projects); the verify phase re-measures over the next N=10 projects (≥3 SPIR) with defined rollback triggers targeting the trims, never the repairs. Per architect ruling, the T13(b) real-spawn probe runs at verify (spawns branch from main HEAD, so pre-merge probes verify the wrong tree).Tests
Full suite: 3,744 passed, 0 failures (196+ files). 8 new/extended enforcement suites; 10 existing suites reworked off deleted paths. No flaky tests skipped.
Review record
~20 CMAP rounds across the phases; every REQUEST_CHANGES either fixed or rebutted with evidence that survived subsequent rounds. Full per-phase rebuttals in
codev/projects/1252-prompt-architecture-single-own/.Refs #1252 (verify phase remains before close)
🤖 Generated with Claude Code