Add managed image generation (/codex:imagegen)#357
Conversation
Fixes openai#356. The companion runs code tasks through a single, serialized codex app-server connection. Image generation never had a managed path: the only working route was a bare parallel Codex CLI, which contends for the same websocket and returns 403/429 while still spending quota. Routing an image prompt through the app-server task runtime instead just sat in the "starting" phase. The hang was not a missing capability. The active provider advertises imageGeneration, image_generation is a stable default-on feature, and a turn does emit an `imageGeneration` item carrying the PNG (base64 `result` plus a `savedPath`). The real problem is that after producing the image the model often keeps going, running shell commands to verify or convert the file, so the turn runs for minutes while the companion waits for it to finish. This adds a managed image path that captures the `imageGeneration` item the moment it arrives, writes the bytes to --out, then interrupts the turn so the post-image tail never runs. Because it reuses the one serialized app-server connection, image jobs never run concurrently, the exact contention the bare-CLI route caused. - /codex:imagegen plus an `imagegen` companion subcommand - --out (write target), --image ref[,ref...] (reference images for editing), --background (tracked job), --force (allow overwrite), --model - a read-only thread: the managed path decodes and writes the bytes itself, so the model needs no workspace write access - a startup timeout (CODEX_PLUGIN_IMAGE_TIMEOUT_MS, default 180s) and fast failure on an empty turn or an error notification - fake-codex coverage for the happy path, the savedPath fallback, no-image, error, timeout, reference inputs, the JSON payload, and overwrite refusal
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9f1b9d1486
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } catch { | ||
| // Ignore handler errors; the timeout is the backstop. | ||
| } |
There was a problem hiding this comment.
When --out cannot be written (for example the parent path is a file, permissions deny the write, or the target is a directory even with --force), saveImageFromItem() throws from inside this notification handler, but this blanket catch swallows it and leaves the promise pending until the 180s timeout. In that scenario the image was actually generated, but the command reports “produced no image” and also never interrupts the turn; settle the promise with the filesystem error instead of ignoring handler exceptions.
Useful? React with 👍 / 👎.
1 participant
Closes #356.
Why
The companion runs code tasks through one serialized codex app-server connection. Image generation never had a managed path. The only working route was a bare parallel Codex CLI, which contends for the same websocket and returns 403/429 while still spending quota. Sending an image prompt through the app-server task runtime instead just sat in the "starting" phase, which is the hang reported in #356.
What it actually was
Not a missing capability.
modelProvider/capabilities/readreportsimageGeneration: true,image_generationis a stable default-on feature, and a turn does emit animageGenerationitem carrying the PNG (base64resultplus asavedPath). The image arrives early. The reason the turn looks hung is that after producing the image the model often keeps going, running shell commands to verify or convert the file, so the turn runs for minutes while the companion waits forturn/completed.The change
A managed image path that captures the
imageGenerationitem the moment it arrives, writes the bytes to--out, then interrupts the turn so the post-image tail never runs. It reuses the single serialized app-server connection, so image jobs never run concurrently, which is the contention the bare-CLI route caused. End to end against a real account this returns in about a minute instead of hanging./codex:imagegenand animagegencompanion subcommand--out,--image ref[,ref...](reference images for editing),--background,--force,--modelCODEX_PLUGIN_IMAGE_TIMEOUT_MS, default 180s) and fast failure on an empty turn or an error notificationTests
node --test. The fake codex grows animagegenbehavior; coverage includes the happy path (asserting the written bytes equal the decoded base64result, not a copy ofsavedPath), thesavedPathfallback, no-image, error, timeout, reference inputs, the--jsonpayload, and overwrite refusal.Happy to adjust the command surface or split this up if you would prefer a different shape.