Skip to content

Commit c916a21

Browse files
committed
feat: snapshot-on-push + npm run rollback
## ELI5 **Problem.** The README documented "rollback" as `git revert + push`. That restores local *content* to a previous git state, but it does **not** restore a known *platform* snapshot — the subsequent push has all the same drift problems, so a "rollback" can clobber unrelated dashboard edits made since the bad deploy. There's also no engine- level record of *what was on the dashboard before the push*, so even in principle you can't put the platform back exactly the way it was. **What this fix does.** Before each PATCH, the engine writes both: - the *outgoing* payload (what we're about to send), AND - the *current platform* payload (what's there right now) to a per-push directory: ``` .vapi-state.<env>.snapshots/<ISO-timestamp>/<resource-type>/<id>.json ``` Snapshots are operator-local (gitignored) and recreated each push. A single timestamped directory pins one push run, so rollback can target an entire push, not individual PATCHes. `npm run rollback -- <env> --to <ISO-timestamp>` re-applies each `platform` payload from the snapshot as a PATCH. `--list` prints all available timestamps. **Outcome you'll notice.** Real undo. After a bad push, run `npm run rollback -- <env> --list` to see your snapshots, pick the one from before the bad push, and `--to <timestamp>` puts the dashboard back to that state. No more "I hope `git revert` does what I want." --- Real undo. Before each PATCH, write the *outgoing* (local) payload AND the *current platform* payload to a per-push directory. `npm run rollback -- <env> --to <ISO-timestamp>` re-applies each platform payload as a PATCH, restoring the dashboard to its state at the moment of the snapshot. Snapshots are operator-local state (.vapi-state.<env>.snapshots/), gitignored, and recreated on every push. Each push pins one timestamped directory so rollback can target an entire push, not individual PATCHes. Files: - src/snapshot.ts (NEW): writeSnapshot, listSnapshotTimestamps, loadSnapshot. Pins a single timestamp per push run (getRunSnapshotDir is idempotent within the process). Reuses sortedKeysReplacer so snapshot files have deterministic key order too. - src/rollback-cmd.ts (NEW): npm run rollback -- <env> --to <timestamp> re-applies each platform payload as a PATCH. --list prints available snapshots. - src/push.ts: writeSnapshot call after drift check passes. Costs one extra GET per resource (acceptable for the safety guarantee — follow- up: plumb drift's GET result through to avoid the duplicate fetch). Snapshot failures don't block the push. - package.json: rollback script. - .gitignore: .vapi-state.*.snapshots/ already covered. - AGENTS.md: document npm run rollback / --list. - tests/snapshot.test.ts: writeSnapshot creates the right path, multi-resource snapshots share a timestamp, listSnapshotTimestamps sorted, loadSnapshot round-trips. Closes improvements.md #3. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
1 parent 05675a0 commit c916a21

7 files changed

Lines changed: 537 additions & 1 deletion

File tree

AGENTS.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -752,6 +752,8 @@ npm run push -- <org> --strict # Abort push if any validator
752752
npm run apply -- <org> # Pull then push (full sync)
753753
npm run validate -- <org> # Lint resources locally (fails fast on schema drift)
754754
npm run sim -- <org> --suite <name> --target <name> # Run a simulation suite against an assistant/squad
755+
npm run rollback -- <org> --to <ISO-timestamp> # Re-apply a snapshot taken before a push
756+
npm run rollback -- <org> --list # List available snapshots
755757
756758
# Testing
757759
npm run call -- <org> -a <assistant-name> # Call an assistant via WebSocket

improvements.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ you which stack PR closes the row.**
5454
| --- | -------------------------------------------------------- | -------------------------------------------------- | ---------- | --------------------------------- |
5555
| 1 | `push` drift detection | Prevent silent overwrites of dashboard edits | #4 | RESOLVED 2026-04-30 (Stack G) |
5656
| 2 | `apply` same-file conflict | `apply` drops concurrent same-file dashboard edits | #4 | Partial — Stack G GET on push |
57-
| 3 | Rollback | Current undo can clobber newer live changes | #4, #5 | Open (Stack H planned) |
57+
| 3 | Rollback | Current undo can clobber newer live changes | #4, #5 | RESOLVED 2026-04-30 (Stack H) |
5858
| 4 | State schema content hashes | Architectural unlock for #1, #2, #3, #6, #7 | None | RESOLVED 2026-04-30 (Stack F) |
5959
| 5 | `push --dry-run` | Cheapest operator-safety win | None | RESOLVED 2026-04-30 (Stack C) |
6060
| 6 | API-level optimistic concurrency | Server-side conflict rejection | Platform | Deferred (Stack I, gated) |

package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
"eval": "tsx src/eval.ts",
1616
"validate": "tsx src/validate-cmd.ts",
1717
"sim": "tsx src/sim-cmd.ts",
18+
"rollback": "tsx src/rollback-cmd.ts",
1819
"build": "tsc --noEmit",
1920
"test": "node --import tsx --test tests/*.test.ts"
2021
},

src/push.ts

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,20 @@ import {
1414
} from "./config.ts";
1515
import { summarizeFindings, validateResources } from "./validate.ts";
1616
import { checkDriftForUpdate } from "./drift.ts";
17+
import { writeSnapshot } from "./snapshot.ts";
18+
19+
// Map a resource label to its state-file key. Used for snapshotting (Stack H)
20+
// — snapshot directories are keyed by the same names the state file uses.
21+
const RESOURCE_LABEL_TO_TYPE: Record<string, ResourceType> = {
22+
tool: "tools",
23+
"structured output": "structuredOutputs",
24+
assistant: "assistants",
25+
squad: "squads",
26+
personality: "personalities",
27+
scenario: "scenarios",
28+
simulation: "simulations",
29+
"simulation suite": "simulationSuites",
30+
};
1731
import {
1832
hashPayload,
1933
loadState,
@@ -88,6 +102,8 @@ async function upsertResourceWithStateRecovery(options: {
88102
// payload, hash it, and compare to lastPulledHash. Refuse to overwrite
89103
// without --overwrite. Skipped in dry-run because the operator just
90104
// wants to see what would happen, and skipped if no baseline hash.
105+
// Stack H — when we successfully fetch the platform payload, snapshot
106+
// it (and our outgoing payload) so `npm run rollback` has a target.
91107
if (!DRY_RUN) {
92108
const stateEntry = stateSection[resourceId];
93109
if (stateEntry) {
@@ -113,6 +129,47 @@ async function upsertResourceWithStateRecovery(options: {
113129
". Continuing.",
114130
);
115131
}
132+
133+
// Snapshot the current platform payload + our outgoing payload to a
134+
// per-push directory so rollback can revert. Costs one extra GET per
135+
// resource — acceptable for the safety guarantee. (Follow-up: plumb
136+
// drift's GET result through to avoid the duplicate fetch.)
137+
try {
138+
const resourceType = RESOURCE_LABEL_TO_TYPE[resourceLabel];
139+
if (resourceType) {
140+
const platformResponse = await fetch(
141+
`${VAPI_BASE_URL}${updateEndpoint}`,
142+
{
143+
method: "GET",
144+
headers: {
145+
Authorization: `Bearer ${process.env.VAPI_TOKEN}`,
146+
},
147+
},
148+
);
149+
if (platformResponse.ok) {
150+
const platformPayloadForSnapshot = await platformResponse.json();
151+
await writeSnapshot({
152+
baseDir: BASE_DIR,
153+
env: VAPI_ENV,
154+
resourceType,
155+
resourceId,
156+
payload: {
157+
outgoing: updatePayload,
158+
platform: platformPayloadForSnapshot,
159+
},
160+
});
161+
}
162+
}
163+
} catch (snapshotErr) {
164+
// Snapshot failures should NOT block the push — the snapshot is a
165+
// safety net, not a precondition. Log and move on.
166+
console.warn(
167+
` ⚠️ snapshot failed for ${resourceLabel} ${resourceId}: ` +
168+
(snapshotErr instanceof Error
169+
? snapshotErr.message
170+
: String(snapshotErr)),
171+
);
172+
}
116173
}
117174
}
118175

src/rollback-cmd.ts

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
// CLI entry: `npm run rollback -- <org> --to <ISO-timestamp>` |
2+
// `npm run rollback -- <org> --list`
3+
//
4+
// Reads .vapi-state.<env>.snapshots/<timestamp>/<resource-type>/<id>.json
5+
// and re-applies the captured *platform* payload via PATCH, restoring the
6+
// dashboard to its state at that snapshot moment.
7+
//
8+
// Self-contained (does not import config.ts) so it can run in isolation
9+
// without triggering the global CLI parser.
10+
11+
import { existsSync, readFileSync } from "fs";
12+
import { dirname, join } from "path";
13+
import { fileURLToPath } from "url";
14+
import {
15+
listSnapshotTimestamps,
16+
loadSnapshot,
17+
} from "./snapshot.ts";
18+
19+
const __dirname = dirname(fileURLToPath(import.meta.url));
20+
const BASE_DIR = join(__dirname, "..");
21+
22+
interface RollbackEnv {
23+
env: string;
24+
token: string;
25+
baseUrl: string;
26+
}
27+
28+
function loadEnvFile(env: string): RollbackEnv {
29+
const envFiles = [
30+
join(BASE_DIR, `.env.${env}`),
31+
join(BASE_DIR, `.env.${env}.local`),
32+
join(BASE_DIR, ".env.local"),
33+
];
34+
const envVars: Record<string, string> = {};
35+
for (const envFile of envFiles) {
36+
if (!existsSync(envFile)) continue;
37+
for (const line of readFileSync(envFile, "utf-8").split("\n")) {
38+
const trimmed = line.trim();
39+
if (!trimmed || trimmed.startsWith("#")) continue;
40+
const eq = trimmed.indexOf("=");
41+
if (eq === -1) continue;
42+
const key = trimmed.slice(0, eq).trim();
43+
let value = trimmed.slice(eq + 1).trim();
44+
if (
45+
(value.startsWith('"') && value.endsWith('"')) ||
46+
(value.startsWith("'") && value.endsWith("'"))
47+
) {
48+
value = value.slice(1, -1);
49+
}
50+
if (envVars[key] === undefined) envVars[key] = value;
51+
}
52+
}
53+
const token = process.env.VAPI_TOKEN || envVars.VAPI_TOKEN;
54+
const baseUrl =
55+
process.env.VAPI_BASE_URL ||
56+
envVars.VAPI_BASE_URL ||
57+
"https://api.vapi.ai";
58+
if (!token) {
59+
console.error(`❌ VAPI_TOKEN not found. Create .env.${env} with VAPI_TOKEN=your-token`);
60+
process.exit(1);
61+
}
62+
return { env, token, baseUrl };
63+
}
64+
65+
function printUsage(): void {
66+
console.error(
67+
[
68+
"Usage:",
69+
" npm run rollback -- <org> --list",
70+
" npm run rollback -- <org> --to <ISO-timestamp>",
71+
"",
72+
"Snapshots are written automatically before each `npm run push` operation",
73+
"to .vapi-state.<env>.snapshots/<timestamp>/. Use --list to inspect available",
74+
"timestamps; use --to <ts> to re-apply the platform payloads from that snapshot.",
75+
].join("\n"),
76+
);
77+
}
78+
79+
const ENDPOINT_MAP: Record<string, string> = {
80+
tools: "/tool",
81+
structuredOutputs: "/structured-output",
82+
assistants: "/assistant",
83+
squads: "/squad",
84+
personalities: "/eval/simulation/personality",
85+
scenarios: "/eval/simulation/scenario",
86+
simulations: "/eval/simulation",
87+
simulationSuites: "/eval/simulation/suite",
88+
evals: "/eval",
89+
};
90+
91+
interface ParsedArgs {
92+
env: string;
93+
list: boolean;
94+
to?: string;
95+
}
96+
97+
function parseArgs(): ParsedArgs {
98+
const args = process.argv.slice(2);
99+
const env = args[0];
100+
if (!env) {
101+
printUsage();
102+
process.exit(1);
103+
}
104+
const SLUG_RE = /^[a-z0-9]([a-z0-9-]*[a-z0-9])?$/;
105+
if (!SLUG_RE.test(env)) {
106+
console.error(`❌ Invalid org name: ${env}`);
107+
process.exit(1);
108+
}
109+
const parsed: ParsedArgs = { env, list: false };
110+
for (let i = 1; i < args.length; i++) {
111+
const a = args[i];
112+
if (a === "--list") parsed.list = true;
113+
else if (a === "--to") parsed.to = args[++i];
114+
else if (a === "--help" || a === "-h") {
115+
printUsage();
116+
process.exit(0);
117+
}
118+
}
119+
if (!parsed.list && !parsed.to) {
120+
console.error("❌ Specify --list or --to <timestamp>");
121+
printUsage();
122+
process.exit(1);
123+
}
124+
return parsed;
125+
}
126+
127+
async function main(): Promise<void> {
128+
const args = parseArgs();
129+
130+
if (args.list) {
131+
const timestamps = await listSnapshotTimestamps(BASE_DIR, args.env);
132+
if (timestamps.length === 0) {
133+
console.log(`No snapshots found for ${args.env}.`);
134+
return;
135+
}
136+
console.log(`Snapshots for ${args.env}:`);
137+
for (const t of timestamps) console.log(` ${t}`);
138+
return;
139+
}
140+
141+
const cfg = loadEnvFile(args.env);
142+
const entries = await loadSnapshot(BASE_DIR, args.env, args.to!);
143+
if (entries.length === 0) {
144+
console.log("Snapshot directory exists but contains no resources.");
145+
return;
146+
}
147+
148+
// We need state so we can resolve resourceId → UUID for the PATCH path.
149+
// Snapshot files don't store the UUID directly because the snapshot is
150+
// keyed by resourceId; the same resourceId points at the same UUID across
151+
// pushes (unless renamed, in which case the snapshot is stale anyway).
152+
const stateFile = join(BASE_DIR, `.vapi-state.${args.env}.json`);
153+
if (!existsSync(stateFile)) {
154+
console.error(`❌ State file not found: ${stateFile}`);
155+
process.exit(1);
156+
}
157+
const state = JSON.parse(readFileSync(stateFile, "utf-8")) as Record<
158+
string,
159+
Record<string, { uuid: string }>
160+
>;
161+
162+
console.log(`🔁 Rollback ${args.env} → snapshot ${args.to}`);
163+
console.log(` ${entries.length} resource(s) to restore\n`);
164+
165+
let restored = 0;
166+
let skipped = 0;
167+
for (const entry of entries) {
168+
const endpoint = ENDPOINT_MAP[entry.resourceType];
169+
if (!endpoint) {
170+
console.warn(` ⚠️ Unknown resource type: ${entry.resourceType}, skipping`);
171+
skipped++;
172+
continue;
173+
}
174+
const section = state[entry.resourceType];
175+
const uuid = section?.[entry.resourceId]?.uuid;
176+
if (!uuid) {
177+
console.warn(
178+
` ⚠️ No UUID in state for ${entry.resourceType}/${entry.resourceId} — skipping`,
179+
);
180+
skipped++;
181+
continue;
182+
}
183+
process.stdout.write(` 🔁 ${entry.resourceType}/${entry.resourceId} ... `);
184+
const response = await fetch(`${cfg.baseUrl}${endpoint}/${uuid}`, {
185+
method: "PATCH",
186+
headers: {
187+
Authorization: `Bearer ${cfg.token}`,
188+
"Content-Type": "application/json",
189+
},
190+
body: JSON.stringify(entry.payload.platform),
191+
});
192+
if (!response.ok) {
193+
const text = await response.text();
194+
console.log(`❌ ${response.status}`);
195+
console.error(` ${text}`);
196+
skipped++;
197+
continue;
198+
}
199+
console.log("✅");
200+
restored++;
201+
}
202+
203+
console.log(
204+
`\n📊 Rollback summary: ${restored} restored, ${skipped} skipped`,
205+
);
206+
if (skipped > 0) process.exit(1);
207+
}
208+
209+
main().catch((error) => {
210+
console.error("\n❌ Rollback failed:", error instanceof Error ? error.message : error);
211+
process.exit(1);
212+
});

0 commit comments

Comments
 (0)