feat(openrouter): surface generated images from assistant responses#1804
Open
jimmiebfulton wants to merge 1 commit into
Open
feat(openrouter): surface generated images from assistant responses#1804jimmiebfulton wants to merge 1 commit into
jimmiebfulton wants to merge 1 commit into
Conversation
OpenRouter image-generation models (e.g. google/gemini-flash-image-preview) return generated images as a sibling `images` array on the assistant message rather than embedding them inside `content`. This change captures those images and converts them into `completion::AssistantContent::Image` blocks alongside the text response, making them accessible to callers without any extra parsing. - Add `ResponseImage` struct (mirrors the `image_url` content-part shape). - Add private `parse_data_uri` helper to split data URIs into (mime, payload). - Add `images: Vec<ResponseImage>` field on `Message::Assistant` with `#[serde(default, skip_serializing)]` — inbound only, never sent in requests. - Extend the `CompletionResponse → completion::CompletionResponse` conversion to append each ResponseImage as an inline base64 block (data URIs) or a URL-sourced image (plain URLs). - Add `images: Vec::new()` to the two `Message::Assistant` construction sites in the `TryFrom` impls so they keep compiling. - Add three tests: base64 data URI extraction, plain URL extraction, and verification that the images field is absent from serialized requests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OpenRouter image-generation models (e.g.
google/gemini-flash-image-preview) return generated images as a siblingimagesarray on the assistant message rather than embedding them insidecontent. Without this change, those images are silently dropped and callers have no way to access them.This PR captures the
imagesarray and converts each entry into acompletion::AssistantContent::Imageblock alongside the text response.Changes
ResponseImagestruct mirroring theimage_urlcontent-part shape (inbound only).parse_data_uriprivate helper to splitdata:<mime>;base64,<payload>URIs.Message::Assistantgainsimages: Vec<ResponseImage>with#[serde(default, skip_serializing)]— received from responses, never serialized back into requests.TryFrom<CompletionResponse>: base64 data URIs become inlineimage_base64blocks; plain URLs become URL-sourcedImageblocks.imagesis absent from serialized requests.AI Assistance
This implementation was developed with significant AI assistance (Claude). The design, tests, and edge cases were reviewed and validated by the author before submission.
Issue
Fixes #807