Skip to content

feat(client): optimistic prompt admission with client-minted IDs - #43520

Merged
kitlangton merged 5 commits into
v2from
prompt-outbox
Aug 20, 2026
Merged

feat(client): optimistic prompt admission with client-minted IDs#43520
kitlangton merged 5 commits into
v2from
prompt-outbox

Conversation

@kitlangton

@kitlangton kitlangton commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

What

Prompt sends become idempotent everywhere and render the instant you hit enter. The data layer gains session.prompt: it POSTs under a client-minted inbox ID, renders the prompt immediately, and lets the durable session.inbox.enqueued echo reconcile by that same ID. No new endpoints, no protocol changes — the server's admission has been idempotent per inbox ID all along; this PR makes the client use that contract.

Every prompt POST now carries a stable client-minted ID, so a user-retried send after a lost response can no longer double-admit — the server returns the original row instead of admitting twice.

Before: submit → POST → wait for the SSE echo (or a pending re-fetch) → the message appears. On a slow connection the composer sits frozen with your text for the whole round-trip, and the first message of a new session shows nothing at all until the POST resolves.

After: submit → the message renders synchronously, the composer clears, and a new session's view opens immediately → the echo confirms the same row in place. A rejection rolls the row back, surfaces the existing toast, and restores your text to the composer (unless you already started typing something new).

How

packages/client/src/solid/data.ts (the only mechanism change)

  • session.prompt(input) — mints SessionMessage.ID.create() when the caller didn't, inserts the optimistic row synchronously, and POSTs with that id.
  • An outbox set tracks in-flight admissions awaiting their echo. Entries clear on the echo or on rollback — not on POST success, which typically precedes the echo. It exists for exactly two reasons:
    • a rejection rolls back only rows this call admitted and the echo has not acknowledged — a late transport failure after the echo, or a failed retry under an already-durable ID, cannot delete server state;
    • pending.sync and message.sync merge in-flight optimistic rows into the fetched lists, so a re-fetch racing the admission (even one landing after the POST resolves but before the echo) cannot wipe a row the server's projection doesn't show yet. Both merges are no-ops when the outbox is empty.
  • The inbox.enqueued and inbox.cancelled handler bodies are extracted into admitLocal/retractLocal, shared by the event path and the optimistic path. admitLocal upserts by inbox ID: the durable echo replaces the optimistic placeholder in place. That upsert is the reconciliation — there is no diffing, no versioning, no merge logic.
  • Files and skills stay off the optimistic row: their durable forms are server-loaded (content, mime, resolution), so they fill in when the echo upserts the row.

packages/tui/src/component/prompt/index.tsx — submission fires data.session.prompt without awaiting: the composer clears and a new session navigates immediately (replacing the 50ms navigation hack); on rejection the failed text is restored to the composer if it is still empty.

packages/app/src/components/prompt-input/submit.tssendFollowupDraft prompts through input.data.session.prompt (it already minted and passed an id, so the app gets retry-idempotency it previously only had by accident of never retrying).

Scope

  • Prompts only. synthetic, compact, and command admissions keep their existing paths — same pattern applies if wanted later.
  • No transport/gap-detection changes: event delivery and reconnect healing are untouched. This deliberately does not attempt "the client always knows when it's wrong"; it makes sends feel instant and retries safe.
  • No auto-retry. A failed send still surfaces to the user; if they resend, the same ID makes it safe.

Testing

  • New tests in packages/tui/test/cli/tui/data.test.tsx (real generated client over a fake fetch, no mocks of the layer under test):
    • optimistic row renders before the server responds; racing pending.sync and message.sync re-fetches (server returns []) preserve it; the durable echo upserts server-loaded data (files, durable time) onto the placeholder; a late 500 after the echo does not delete acknowledged state;
    • the POST resolving before the echo does not reopen the wipe window;
    • a server rejection rolls back pending, input, and the transcript row, and propagates the error;
    • a retry under the same client-minted ID produces exactly one row after two POSTs, and a failed retry after the row is acknowledged does not roll back server state.
  • Suites: tui 743/0, app 44/0, client typecheck green; full-repo typecheck 33/33 packages.
  • Live end-to-end: real serve process with a fake OpenAI-compatible model, TUI connected through a proxy injecting 2.5s latency and 500s on POST .../prompt. Verified: prompts render instantly on both new and existing sessions, rejection rolls back and restores the composer, and resend succeeds. This testing is what caught (and fixed) the composer double-display, the new-session dead window, and the message.sync wipe race.

Demo

Recorded against a real server through the flaky proxy (2.5s injected latency; captions note each phase).

Before — stock v2 under 2.5s latency (enter, then nothing until the round-trip completes):

before.mp4

After — this PR, same latency (instant render; injected 500 → rollback + composer restore; instant resend):

after.mp4

Flow

sequenceDiagram
  participant UI as TUI / App
  participant Data as data.session.prompt
  participant Server

  UI->>Data: prompt(text)
  Data->>Data: mint msg_id, render row (pending + input + transcript)
  Data->>Server: POST /prompt { id: msg_id }
  Server-->>Data: session.inbox.enqueued { inboxID: msg_id }
  Data->>Data: echo upserts by id (durable payload replaces the placeholder)
  Note over Data,Server: rejection before echo → roll back row, toast, restore composer<br/>failure after echo → row stays (it is server state)<br/>retry → same id → idempotent admission, no duplicate
Loading

The data layer's new session.prompt renders the prompt immediately under
a client-minted inbox ID and lets the durable inbox.enqueued echo
reconcile by that same ID. A rejection rolls back only unacknowledged
rows, a racing pending re-fetch preserves in-flight admissions, and
retries reuse the same ID so the server's idempotent admission cannot
double-admit. TUI and web submit through it.
Review-pass fixes from the simplify skill's three-agent review:

- admitLocal upserts instead of deduping, so the durable enqueued echo
  replaces the optimistic placeholder (server-loaded files and durable
  timeCreated land instead of being silently ignored)
- outbox entries clear on the echo or rollback, not on POST success,
  closing a window where a racing pending re-fetch could wipe the row
  between response and echo
- prompt() admits optimistically only for fresh IDs, so a failed retry
  under an already-acknowledged ID cannot roll back server state
- rollback is reachable even when the client throws synchronously
- removeSession sweeps outbox entries for the deleted session
- admitLocal/retractLocal batch their store writes
- tests: echo upsert asserted with durable-only data (files, time),
  new POST-before-echo race test, failed-retry-after-ack case
Live end-to-end testing against a real server behind a flaky proxy
exposed two gaps the unit tests could not see:

- the submit flow awaited the prompt POST before clearing the composer,
  so under network latency the message displayed twice (transcript +
  composer) for the whole round-trip, and a brand-new session showed
  nothing at all because navigation was gated on the same await
- message.sync reconciled the server list wholesale, so a re-fetch
  racing an optimistic admission (guaranteed by immediate navigation)
  wiped the in-flight transcript row until the echo restored it

Submission now fires the prompt without awaiting: the composer clears
and the session view opens immediately (replacing the 50ms navigation
hack). On rejection the data layer rolls the row back, a toast surfaces
the error, and the failed text is restored to the composer unless the
user already started typing. message.sync gains the same in-flight
merge as pending.sync, with test coverage.
Optimistic rendering, immediate navigation, and rollback-with-restore
now sit behind the optimistic_prompt experiment (Experiments dialog,
default off). With the flag off the submit flow keeps the previous
awaited behavior, including the deferred navigation for new sessions.

Client-minted prompt IDs stay unconditional: the POST always carries a
stable ID, so the server's idempotent admission prevents duplicate
sends on retry regardless of the experiment. data.session.prompt only
admits optimistically when the caller opts in; the web app stays on the
non-optimistic path.

Verified live against a real server behind a latency/failure proxy:
flag off reproduces stock behavior, flag on renders instantly, and the
config watcher applies the toggle without restart.
@kitlangton kitlangton changed the title feat(client): optimistic prompt admission with client-minted IDs feat(client): idempotent prompt sends with opt-in optimistic rendering Aug 19, 2026
@kitlangton kitlangton changed the title feat(client): idempotent prompt sends with opt-in optimistic rendering feat(client): optimistic prompt admission with client-minted IDs Aug 19, 2026
@kitlangton
kitlangton marked this pull request as ready for review August 20, 2026 00:26
@kitlangton
kitlangton merged commit 6b09b9e into v2 Aug 20, 2026
10 checks passed
@kitlangton
kitlangton deleted the prompt-outbox branch August 20, 2026 00:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant