Skip to content

feat(embeddings): expose token usage via embed_texts_with_usage#1791

Merged
gold-silver-copper merged 3 commits into
0xPlaygrounds:mainfrom
sergiomeneses:feat/embedding-usage
May 29, 2026
Merged

feat(embeddings): expose token usage via embed_texts_with_usage#1791
gold-silver-copper merged 3 commits into
0xPlaygrounds:mainfrom
sergiomeneses:feat/embedding-usage

Conversation

@sergiomeneses
Copy link
Copy Markdown
Contributor

Summary

Add EmbeddingResponse struct and embed_texts_with_usage / embed_text_with_usage methods to the EmbeddingModel trait, enabling consumers to track token consumption when generating embeddings.

Currently embed_texts() returns only Vec<Embedding>, discarding provider-supplied token usage info.

Changes

  • Trait: Add EmbeddingResponse struct (embeddings + usage) and embed_texts_with_usage() / embed_text_with_usage() to EmbeddingModel
  • OpenAI: Override embed_texts_with_usage() with real usage from API response
  • VoyageAI: Override embed_texts_with_usage() with real usage from API response
  • EmbeddingsBuilder: Add build_with_usage() accumulating usage across all batches
  • Backward-compatible: Existing embed_texts(), embed_text(), build() unchanged

Usage

// Direct with usage tracking
let response = model.embed_texts_with_usage(docs).await?;
println!("Tokens used: {}", response.usage.input_tokens);

// Via builder with usage tracking
let builder = EmbeddingsBuilder::new(model);
let (pairs, total_usage) = builder.build_with_usage().await?;

Related: #988
`

Add EmbeddingResponse struct and embed_texts_with_usage /
embed_text_with_usage methods to the EmbeddingModel trait,
enabling consumers to track token consumption when generating
embeddings.

- EmbeddingResponse wraps Vec<Embedding> + Usage
- Override embed_texts_with_usage in OpenAI and VoyageAI
- Add EmbeddingsBuilder::build_with_usage() accumulating
  usage across all batches
- Existing embed_texts/embed_text/build APIs unchanged
  (backward-compatible defaults)
@sergiomeneses
Copy link
Copy Markdown
Contributor Author

do you guys needs someting else to progress with this?

@gold-silver-copper gold-silver-copper added this pull request to the merge queue May 29, 2026
Merged via the queue into 0xPlaygrounds:main with commit a92beee May 29, 2026
5 checks passed
@github-actions github-actions Bot mentioned this pull request May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants