Skip to content

Commit 2eaf382

Browse files
committed
feat: drift detection on push (--overwrite to bypass)
**Problem.** Today: you pull, your teammate edits the same assistant on the dashboard during a live test, you push your unrelated branch, and their dashboard edit disappears with no warning. Customer-success reps update business hours via the dashboard; the next gitops push silently reverts them. Even `git revert + push` rollbacks have the same problem — they overwrite whatever's currently live, not just the change being reverted. The engine had no way to detect this because the state file only stored name→UUID, no record of the platform's content at last pull. **What this fix does.** Now that Stack F populates `lastPulledHash`, drift detection becomes possible. Before each PATCH, the engine GETs the current platform payload, hashes it, and compares to the `lastPulledHash` in state. - Hashes match → continue silently. - Hashes differ + no flag → **refuse the push**, point at the drift, ask the operator to either pull-and-resolve or pass `--overwrite` to take ownership. - Hashes differ + `--overwrite` → log "overwriting drift" and proceed. - No baseline (legacy state, first push after Stack F) → log "drift unknown — proceeding" and don't block. Also adds a specific helper for the **Cartesia voice picker** footgun: if `pronunciationDictId` was set at last pull but isn't on the platform now, surface that explicitly so the operator notices. **Outcome you'll notice.** Concurrent dashboard edits no longer disappear silently. If someone else touched a resource between your pull and your push, you see the conflict at push time and have to make an explicit call (overwrite, or pull and resolve). The engine becomes a real safety rail rather than a blind PATCH machine. --- Before each PATCH, GET the current platform payload, hash it, and compare to the lastPulledHash recorded in state (Stack F). If the hashes differ, the dashboard has drifted away from the version we last pulled — refuse to push without --overwrite. Behavior matrix: - No lastPulledHash (legacy state, first push after Stack F): log "drift unknown — proceeding" and continue. Don't block. - Hashes match: continue silently. - Hashes differ + no --overwrite: refuse the push, return null. - Hashes differ + --overwrite: log "overwriting drift" and continue. Files: - src/drift.ts (NEW): checkDriftForUpdate(endpoint, state, overwrite). GETs platform, strips server-managed fields (id/orgId/createdAt/etc) to align hash basis with cleanResource()'s output, sha256 compares. Returns DriftCheckResult with reason and message for caller logging. - src/state-serialize.ts: checkPronunciationDictDrop helper for the Cartesia voice-picker case (improvements.md #7) — pure data, safe to import in tests. - src/config.ts: --overwrite flag. - src/push.ts: drift gate in upsertResourceWithStateRecovery before every PATCH. Skipped in dry-run (operator wants to see what would happen). Skipped if no baseline. - tests/drift.test.ts: hash-match → ok, hash-differ-no-overwrite → ok=false, hash-differ-overwrite → ok=true, no-baseline → ok=true. Closes improvements.md #1, #7. Partial #2 (push side caught; pull side same-file conflict still requires manual resolution). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- ## Update — 11labs `pronunciationDictionaryLocators` array also covered `checkPronunciationDictDrop` now detects drops in both pronunciation- dictionary shapes Vapi exposes: - **11labs** (the documented shape): `voice.pronunciationDictionaryLocators[]` — array of `{ pronunciationDictionaryId, versionId }`. We warn on N → M shrinks (M < N) including N → 0 and array-going-missing. - **Cartesia** (passthrough — not in Vapi docs but observed): `voice.pronunciationDictId` — single string id. Existing 1 → 0 detection unchanged. Reference: https://docs.vapi.ai/assistants/pronunciation-dictionaries Six new test cases pin the 11labs behavior: array clear (1 → 0), shrink (2 → 1), array-going-missing entirely, no-op when unchanged, no-op when locators are added (additive growth shouldn't warn), and the defensive hybrid case where a payload carries both shapes.
1 parent f0fe2ca commit 2eaf382

6 files changed

Lines changed: 384 additions & 4 deletions

File tree

improvements.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -52,13 +52,13 @@ you which stack PR closes the row.**
5252

5353
| # | Title | Why it matters | Depends on | Status |
5454
| --- | -------------------------------------------------------- | -------------------------------------------------- | ---------- | --------------------------------- |
55-
| 1 | `push` drift detection | Prevent silent overwrites of dashboard edits | #4 | Open (Stack G planned) |
56-
| 2 | `apply` same-file conflict | `apply` drops concurrent same-file dashboard edits | #4 | Open (Stack G planned) |
55+
| 1 | `push` drift detection | Prevent silent overwrites of dashboard edits | #4 | RESOLVED 2026-04-30 (Stack G) |
56+
| 2 | `apply` same-file conflict | `apply` drops concurrent same-file dashboard edits | #4 | Partial — Stack G GET on push |
5757
| 3 | Rollback | Current undo can clobber newer live changes | #4, #5 | Open (Stack H planned) |
5858
| 4 | State schema content hashes | Architectural unlock for #1, #2, #3, #6, #7 | None | RESOLVED 2026-04-30 (Stack F) |
5959
| 5 | `push --dry-run` | Cheapest operator-safety win | None | RESOLVED 2026-04-30 (Stack C) |
6060
| 6 | API-level optimistic concurrency | Server-side conflict rejection | Platform | Deferred (Stack I, gated) |
61-
| 7 | Voice edits drop pronunciation-dictionary attachments | Silent regression on Cartesia + 11labs voice edits | #4 | Open (Stack G planned) |
61+
| 7 | Voice edits drop pronunciation-dictionary attachments | Silent regression on Cartesia + 11labs voice edits | #4 | RESOLVED 2026-04-30 (Stack G) |
6262
| 8 | Dashboard prompt edits can in-place duplicate the prompt | Two stacked prompt versions = stitched output | None | Partial — Stack D heuristic |
6363
| 9 | Provider-specific voice schema mismatch (push 400) | `voice.speed` vs `voice.generationConfig.speed` | None | RESOLVED 2026-04-30 (Stack D + A) |
6464
| 10 | Targeted assistant push mints duplicate tools | Re-pushing assistant duplicates `end-call-*` tools | #4 | Partial |

src/config.ts

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ function parseFlags(): {
8787
bootstrapSync: boolean;
8888
dryRun: boolean;
8989
strictValidation: boolean;
90+
overwriteDrift: boolean;
9091
applyFilter: ApplyFilter;
9192
} {
9293
const args = process.argv.slice(3);
@@ -95,12 +96,14 @@ function parseFlags(): {
9596
bootstrapSync: boolean;
9697
dryRun: boolean;
9798
strictValidation: boolean;
99+
overwriteDrift: boolean;
98100
applyFilter: ApplyFilter;
99101
} = {
100102
forceDelete: args.includes("--force"),
101103
bootstrapSync: args.includes("--bootstrap"),
102104
dryRun: args.includes("--dry-run"),
103105
strictValidation: args.includes("--strict"),
106+
overwriteDrift: args.includes("--overwrite"),
104107
applyFilter: {},
105108
};
106109

@@ -115,7 +118,8 @@ function parseFlags(): {
115118
arg === "--force" ||
116119
arg === "--bootstrap" ||
117120
arg === "--dry-run" ||
118-
arg === "--strict"
121+
arg === "--strict" ||
122+
arg === "--overwrite"
119123
)
120124
continue;
121125

@@ -252,6 +256,7 @@ export const {
252256
bootstrapSync: BOOTSTRAP_SYNC,
253257
dryRun: DRY_RUN,
254258
strictValidation: STRICT_VALIDATION,
259+
overwriteDrift: OVERWRITE_DRIFT,
255260
applyFilter: APPLY_FILTER,
256261
} = parseFlags();
257262

src/drift.ts

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
// ─────────────────────────────────────────────────────────────────────────────
2+
// Drift detection — Stack G
3+
//
4+
// Before each PATCH, GET the current platform payload, hash it, and compare
5+
// to the `lastPulledHash` recorded in state. If the hashes differ, the
6+
// dashboard has drifted away from the version we last pulled — refuse to
7+
// push without `--overwrite` (improvements.md #1, #2, #7).
8+
//
9+
// Behavior matrix:
10+
// - No `lastPulledHash` (e.g., legacy state, first push after Stack F):
11+
// log "drift unknown — proceeding" and continue. Don't block.
12+
// - Hashes match: continue silently.
13+
// - Hashes differ + no --overwrite: refuse the push, return false.
14+
// - Hashes differ + --overwrite: log "overwriting drift" and continue.
15+
//
16+
// The check fires GET against the same endpoint the apply function would
17+
// PATCH. We don't centralize it inside `vapiRequest` because POST (create)
18+
// has nothing to compare against — only PATCH (update) is drift-sensitive.
19+
// ─────────────────────────────────────────────────────────────────────────────
20+
21+
import { hashPayload } from "./state-serialize.ts";
22+
import { VAPI_BASE_URL, VAPI_TOKEN } from "./config.ts";
23+
import type { ResourceState } from "./types.ts";
24+
25+
export interface DriftCheckResult {
26+
ok: boolean;
27+
reason: "no-baseline" | "match" | "drift-overwritten" | "drift-blocked";
28+
message?: string;
29+
// Hash of the *current* platform payload — caller may want to update
30+
// state's `lastPulledHash` after a successful push so subsequent pushes
31+
// start from the platform's current state, not the stale pre-overwrite hash.
32+
platformHash?: string;
33+
}
34+
35+
async function fetchPlatformPayload(
36+
endpoint: string,
37+
): Promise<unknown | null> {
38+
// GET against the same path the PATCH would target. 404 means the resource
39+
// was deleted on the dashboard — let the upsert path handle it (the existing
40+
// 404 → "stale mapping, drop and skip" recovery in
41+
// upsertResourceWithStateRecovery covers this case).
42+
const response = await fetch(`${VAPI_BASE_URL}${endpoint}`, {
43+
method: "GET",
44+
headers: { Authorization: `Bearer ${VAPI_TOKEN}` },
45+
});
46+
if (response.status === 404) return null;
47+
if (!response.ok) {
48+
const text = await response.text();
49+
throw new Error(`Drift GET ${endpoint}${response.status}: ${text}`);
50+
}
51+
return response.json();
52+
}
53+
54+
// Strip server-managed fields before hashing so the platform's payload hash
55+
// matches the last-pulled-hash basis (which excluded them via cleanResource).
56+
const SERVER_FIELDS = new Set([
57+
"id",
58+
"orgId",
59+
"createdAt",
60+
"updatedAt",
61+
"analyticsMetadata",
62+
"isDeleted",
63+
"isServerUrlSecretSet",
64+
"workflowIds",
65+
]);
66+
67+
function stripServerFields(payload: unknown): unknown {
68+
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
69+
return payload;
70+
}
71+
const out: Record<string, unknown> = {};
72+
for (const [k, v] of Object.entries(payload as Record<string, unknown>)) {
73+
if (!SERVER_FIELDS.has(k)) out[k] = v;
74+
}
75+
return out;
76+
}
77+
78+
export async function checkDriftForUpdate(options: {
79+
endpoint: string; // e.g. "/assistant/<uuid>"
80+
resourceLabel: string; // for log lines
81+
resourceId: string; // local resource id
82+
state: ResourceState;
83+
overwrite: boolean;
84+
}): Promise<DriftCheckResult> {
85+
const { endpoint, resourceLabel, resourceId, state, overwrite } = options;
86+
87+
if (!state.lastPulledHash) {
88+
return {
89+
ok: true,
90+
reason: "no-baseline",
91+
message:
92+
` ⚠️ drift check skipped for ${resourceLabel} ${resourceId}: ` +
93+
`no lastPulledHash in state. Run \`npm run pull\` to establish a baseline.`,
94+
};
95+
}
96+
97+
const remote = await fetchPlatformPayload(endpoint);
98+
if (remote === null) {
99+
// Resource was deleted on the dashboard — defer to the upsert recovery
100+
// path. Drift is not the right framing here.
101+
return { ok: true, reason: "no-baseline" };
102+
}
103+
104+
const platformHash = hashPayload(stripServerFields(remote));
105+
if (platformHash === state.lastPulledHash) {
106+
return { ok: true, reason: "match", platformHash };
107+
}
108+
109+
if (overwrite) {
110+
return {
111+
ok: true,
112+
reason: "drift-overwritten",
113+
platformHash,
114+
message:
115+
` ⚠️ drift on ${resourceLabel} ${resourceId}: platform changed since last pull, ` +
116+
`overwriting (--overwrite).`,
117+
};
118+
}
119+
120+
return {
121+
ok: false,
122+
reason: "drift-blocked",
123+
platformHash,
124+
message:
125+
` ❌ drift detected on ${resourceLabel} ${resourceId}: ` +
126+
`platform hash (${platformHash.slice(0, 8)}...) differs from last-pulled ` +
127+
`(${state.lastPulledHash.slice(0, 8)}...). ` +
128+
`Re-run pull, resolve locally, or push with --overwrite to take ownership.`,
129+
};
130+
}
131+
132+
// Re-export the pure helper from state-serialize so call sites can import
133+
// from drift.ts but tests can import the pure version directly.
134+
export { checkPronunciationDictDrop } from "./state-serialize.ts";

src/push.ts

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,13 @@ import {
77
FORCE_DELETE,
88
DRY_RUN,
99
STRICT_VALIDATION,
10+
OVERWRITE_DRIFT,
1011
APPLY_FILTER,
1112
BASE_DIR,
1213
removeExcludedKeys,
1314
} from "./config.ts";
1415
import { summarizeFindings, validateResources } from "./validate.ts";
16+
import { checkDriftForUpdate } from "./drift.ts";
1517
import {
1618
hashPayload,
1719
loadState,
@@ -82,6 +84,38 @@ async function upsertResourceWithStateRecovery(options: {
8284
` 🔄 Updating ${resourceLabel}: ${resourceId} (${existingUuid})`,
8385
);
8486

87+
// Stack G — drift detection. Before PATCH, GET the current platform
88+
// payload, hash it, and compare to lastPulledHash. Refuse to overwrite
89+
// without --overwrite. Skipped in dry-run because the operator just
90+
// wants to see what would happen, and skipped if no baseline hash.
91+
if (!DRY_RUN) {
92+
const stateEntry = stateSection[resourceId];
93+
if (stateEntry) {
94+
try {
95+
const drift = await checkDriftForUpdate({
96+
endpoint: updateEndpoint,
97+
resourceLabel,
98+
resourceId,
99+
state: stateEntry,
100+
overwrite: OVERWRITE_DRIFT,
101+
});
102+
if (drift.message) {
103+
if (drift.ok) console.log(drift.message);
104+
else console.error(drift.message);
105+
}
106+
if (!drift.ok) return null;
107+
} catch (driftErr) {
108+
// A drift check failure should NOT block the push — the existing
109+
// PATCH path will surface the real error. Log and move on.
110+
console.warn(
111+
` ⚠️ drift check failed for ${resourceLabel} ${resourceId}: ` +
112+
(driftErr instanceof Error ? driftErr.message : String(driftErr)) +
113+
". Continuing.",
114+
);
115+
}
116+
}
117+
}
118+
85119
try {
86120
await vapiRequest("PATCH", updateEndpoint, updatePayload);
87121
return existingUuid;

src/state-serialize.ts

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,3 +89,68 @@ export function upsertState(
8989
...patch,
9090
};
9191
}
92+
93+
// Pronunciation-dictionary drop check (improvements.md #7). Detects when a
94+
// dictionary attachment disappears from the platform between pulls. Two
95+
// shapes are supported because Vapi exposes a different field per provider:
96+
//
97+
// - 11labs (documented at
98+
// https://docs.vapi.ai/assistants/pronunciation-dictionaries):
99+
// `voice.pronunciationDictionaryLocators` — array of
100+
// { pronunciationDictionaryId, versionId }. Dashboard edits that
101+
// change the voice can drop entries from this array.
102+
//
103+
// - Cartesia (passthrough; not in Vapi docs but observed in real customer
104+
// payloads): `voice.pronunciationDictId` — single string id. The
105+
// Cartesia voice-picker silently drops the field on voice change.
106+
//
107+
// Pure-data (no network) so safe to import in tests.
108+
type VoiceLike = {
109+
voice?: {
110+
pronunciationDictId?: unknown;
111+
pronunciationDictionaryLocators?: unknown;
112+
};
113+
};
114+
115+
function locatorsArray(value: unknown): unknown[] {
116+
return Array.isArray(value) ? value : [];
117+
}
118+
119+
export function checkPronunciationDictDrop(
120+
resourceId: string,
121+
priorPayload: unknown,
122+
newPayload: unknown,
123+
): string | null {
124+
const priorVoice = (priorPayload as VoiceLike | undefined)?.voice;
125+
const newVoice = (newPayload as VoiceLike | undefined)?.voice;
126+
127+
// Cartesia single-id form. Drops 1 → 0.
128+
if (
129+
priorVoice?.pronunciationDictId &&
130+
typeof priorVoice.pronunciationDictId === "string" &&
131+
!newVoice?.pronunciationDictId
132+
) {
133+
return (
134+
` ⚠️ ${resourceId}: voice.pronunciationDictId was "${priorVoice.pronunciationDictId}" ` +
135+
`at last pull but is missing on platform now. ` +
136+
`Cartesia voice picker drops this silently — re-attach if needed.`
137+
);
138+
}
139+
140+
// 11labs locator-array form. Catches array clears (N → 0) and shrinks
141+
// (N → M, M < N). A drop from 0 → 0 (or undefined → undefined) is a
142+
// no-op and rightly returns null.
143+
const priorLocators = locatorsArray(priorVoice?.pronunciationDictionaryLocators);
144+
if (priorLocators.length > 0) {
145+
const newLocators = locatorsArray(newVoice?.pronunciationDictionaryLocators);
146+
if (newLocators.length < priorLocators.length) {
147+
return (
148+
` ⚠️ ${resourceId}: voice.pronunciationDictionaryLocators dropped from ` +
149+
`${priorLocators.length} entry/entries at last pull to ${newLocators.length} on platform now. ` +
150+
`11labs dashboard voice edits can drop these silently — re-attach if needed.`
151+
);
152+
}
153+
}
154+
155+
return null;
156+
}

0 commit comments

Comments
 (0)