Skip to content

fix(langchain): emit cache creation tokens #4261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dvirski merged 3 commits into from
Jun 17, 2026
+15 −3
Merged

fix(langchain): emit cache creation tokens #4261

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
28c10df
fix(langchain):emit cache creation tokens
dvirski Jun 10, 2026
3cbc245
use opentel semconv cache attributes consts
dvirski Jun 16, 2026
bc84aaf
emitting 0 for cache tokens + pr comment fix
dvirski Jun 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions ...telemetry-instrumentation-langchain/opentelemetry/instrumentation/langchain/span_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,7 +393,8 @@ def set_chat_response_usage(
input_tokens = 0
output_tokens = 0
total_tokens = 0
cache_read_tokens = 0
cache_read_tokens = None
cache_creation_tokens = None

# Early return if no generations to avoid potential issues
if not response.generations:
Expand Down Expand Up @@ -423,7 +424,12 @@ def set_chat_response_usage(
input_token_details = generation.message.usage_metadata.get(
"input_token_details", {}
)
cache_read_tokens += input_token_details.get("cache_read", 0)
raw_cache_read = input_token_details.get("cache_read")
if isinstance(raw_cache_read, (int, float)):
cache_read_tokens = (cache_read_tokens or 0) + raw_cache_read
raw_cache_creation = input_token_details.get("cache_creation")
if isinstance(raw_cache_creation, (int, float)):
cache_creation_tokens = (cache_creation_tokens or 0) + raw_cache_creation

@nina-kollman nina-kollman Jun 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not init the 'cache_creation_tokens' to 0 ? same for cache_read_tokens

@dvirski dvirski Jun 16, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cache_creation_tokens and cache_read_tokens might not be provided at all. so we want to distinguish between not provided and 0.

If not provided we omit them, but we do want to log them if they have value of 0.

except Exception as e:
# If there's any issue processing usage metadata, continue without it
logger.warning("Error processing usage metadata: %s", e)
Expand All @@ -433,7 +439,8 @@ def set_chat_response_usage(
input_tokens > 0
or output_tokens > 0
or total_tokens > 0
or cache_read_tokens > 0
or cache_read_tokens is not None
or cache_creation_tokens is not None
):
_set_span_attribute(
span,
Expand All @@ -455,6 +462,11 @@ def set_chat_response_usage(
GenAIAttributes.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,
cache_read_tokens,
)
_set_span_attribute(
span,
GenAIAttributes.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,
cache_creation_tokens,
)
if record_token_usage:
vendor = span.attributes.get(GenAIAttributes.GEN_AI_PROVIDER_NAME, "langchain")

Expand Down
Loading