feat(pinecone): add Inference API instrumentation for embed and rerank#4269
Open
by-Kaimercer wants to merge 1 commit into
Open
feat(pinecone): add Inference API instrumentation for embed and rerank#4269by-Kaimercer wants to merge 1 commit into
by-Kaimercer wants to merge 1 commit into
Conversation
Adds OpenTelemetry tracing for Pinecone's Inference API calls: - pc.inference.embed() — generates vector embeddings - pc.inference.rerank() — reranks documents using cross-encoder models New span attributes: - gen_ai.request.model — the embedding/reranking model used - gen_ai.usage.input_count — number of input texts/documents - gen_ai.usage.output_count — number of embeddings returned - pinecone.embedding.dimensionality — dimension of embedding vectors - pinecone.rerank.top_n — top_n parameter for reranking - pinecone.rerank.result_count — number of reranked results - pinecone.usage.read_units — read units consumed Spans use GenAI semantic conventions (gen_ai.*) matching the pattern used by other LLM provider instrumentations (OpenAI, Anthropic, etc.). Bumped version to 0.62.0. Fixes traceloop#1618
|
Ayaan Khann seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
📝 WalkthroughWalkthroughThe Pinecone instrumentation package is extended to trace ChangesPinecone Inference Instrumentation
Sequence Diagram(s)sequenceDiagram
participant App
participant PineconeInferenceWrapper
participant _set_inference_input_attributes
participant PineconeInference
participant _set_inference_response_attributes
participant OTelSpan
App->>PineconeInferenceWrapper: embed(model, inputs) / rerank(model, query, documents, top_n)
PineconeInferenceWrapper->>OTelSpan: start span (pinecone.inference.embed / pinecone.inference.rerank)
PineconeInferenceWrapper->>_set_inference_input_attributes: kwargs
_set_inference_input_attributes->>OTelSpan: set model, input_count, query, top_n
PineconeInferenceWrapper->>PineconeInference: call original method
PineconeInference-->>PineconeInferenceWrapper: response
PineconeInferenceWrapper->>_set_inference_response_attributes: response
_set_inference_response_attributes->>OTelSpan: set vector_count, dimensionality / rerank_count, read_units
PineconeInferenceWrapper-->>App: response
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@packages/opentelemetry-instrumentation-pinecone/opentelemetry/instrumentation/pinecone/__init__.py`:
- Around line 129-139: The Pinecone-specific span attributes used in the
set_span_attribute calls are incorrectly sourced from SpanAttributes instead of
AISpanAttributes. Replace SpanAttributes.PINECONE_EMBEDDING_DIMENSIONALITY with
AISpanAttributes.PINECONE_EMBEDDING_DIMENSIONALITY in the first
set_span_attribute call, and replace SpanAttributes.PINECONE_RERANK_RESULT_COUNT
with AISpanAttributes.PINECONE_RERANK_RESULT_COUNT in the second
set_span_attribute call. This ensures Pinecone-specific attributes are correctly
imported and referenced from the appropriate attributes class.
- Line 112: The call to set_span_attribute with
SpanAttributes.PINECONE_RERANK_TOP_N on line 112 is using the wrong attributes
class. Replace SpanAttributes with AISpanAttributes in the set_span_attribute
call for PINECONE_RERANK_TOP_N, since Pinecone-specific attributes are defined
in AISpanAttributes from opentelemetry.semconv_ai (as already correctly used in
lines 167-170 for PINECONE_USAGE_READ_UNITS and PINECONE_USAGE_WRITE_UNITS), not
in SpanAttributes from opentelemetry.semconv.trace. This will prevent an
AttributeError at runtime when the span attribute is accessed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: fb1c6386-ebf8-4703-a094-71d3cf4b5f76
📒 Files selected for processing (2)
packages/opentelemetry-instrumentation-pinecone/opentelemetry/instrumentation/pinecone/__init__.pypackages/opentelemetry-instrumentation-pinecone/opentelemetry/instrumentation/pinecone/version.py
|
|
||
| top_n = kwargs.get("top_n") | ||
| if top_n is not None: | ||
| set_span_attribute(span, SpanAttributes.PINECONE_RERANK_TOP_N, top_n) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify the attribute exists in semconv_ai but not in standard semconv
echo "=== Checking AISpanAttributes for PINECONE_RERANK_TOP_N ==="
rg -n "PINECONE_RERANK_TOP_N" --type py
echo "=== Checking semconv_ai SpanAttributes definition ==="
python -c "from opentelemetry.semconv_ai import SpanAttributes; print([a for a in dir(SpanAttributes) if 'PINECONE' in a])" 2>/dev/null || echo "Could not import semconv_ai"
echo "=== Checking standard semconv SpanAttributes ==="
python -c "from opentelemetry.semconv.trace import SpanAttributes; print([a for a in dir(SpanAttributes) if 'PINECONE' in a])" 2>/dev/null || echo "Could not import semconv.trace"Repository: traceloop/openllmetry
Length of output: 440
🏁 Script executed:
# Check the imports and the relevant code sections
head -50 packages/opentelemetry-instrumentation-pinecone/opentelemetry/instrumentation/pinecone/__init__.py | grep -E "^from|^import"Repository: traceloop/openllmetry
Length of output: 997
🏁 Script executed:
# Read the _set_inference_input_attributes function around line 112
sed -n '88,115p' packages/opentelemetry-instrumentation-pinecone/opentelemetry/instrumentation/pinecone/__init__.pyRepository: traceloop/openllmetry
Length of output: 1004
🏁 Script executed:
# Check lines 167, 170 mentioned in the review
sed -n '165,175p' packages/opentelemetry-instrumentation-pinecone/opentelemetry/instrumentation/pinecone/__init__.pyRepository: traceloop/openllmetry
Length of output: 448
Change line 112 to use AISpanAttributes instead of SpanAttributes.
SpanAttributes from opentelemetry.semconv.trace does not contain Pinecone-specific attributes like PINECONE_RERANK_TOP_N. These attributes are defined in AISpanAttributes from opentelemetry.semconv_ai, as correctly used in lines 167-170 for PINECONE_USAGE_READ_UNITS and PINECONE_USAGE_WRITE_UNITS. Accessing SpanAttributes.PINECONE_RERANK_TOP_N will raise an AttributeError at runtime.
Fix
- set_span_attribute(span, SpanAttributes.PINECONE_RERANK_TOP_N, top_n)
+ set_span_attribute(span, AISpanAttributes.PINECONE_RERANK_TOP_N, top_n)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| set_span_attribute(span, SpanAttributes.PINECONE_RERANK_TOP_N, top_n) | |
| set_span_attribute(span, AISpanAttributes.PINECONE_RERANK_TOP_N, top_n) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/opentelemetry-instrumentation-pinecone/opentelemetry/instrumentation/pinecone/__init__.py`
at line 112, The call to set_span_attribute with
SpanAttributes.PINECONE_RERANK_TOP_N on line 112 is using the wrong attributes
class. Replace SpanAttributes with AISpanAttributes in the set_span_attribute
call for PINECONE_RERANK_TOP_N, since Pinecone-specific attributes are defined
in AISpanAttributes from opentelemetry.semconv_ai (as already correctly used in
lines 167-170 for PINECONE_USAGE_READ_UNITS and PINECONE_USAGE_WRITE_UNITS), not
in SpanAttributes from opentelemetry.semconv.trace. This will prevent an
AttributeError at runtime when the span attribute is accessed.
Comment on lines
+129
to
+139
| set_span_attribute( | ||
| span, | ||
| SpanAttributes.PINECONE_EMBEDDING_DIMENSIONALITY, | ||
| len(first_emb.values), | ||
| ) | ||
|
|
||
| # For rerank responses | ||
| if hasattr(response, "results") and response.results: | ||
| set_span_attribute( | ||
| span, SpanAttributes.PINECONE_RERANK_RESULT_COUNT, len(response.results) | ||
| ) |
There was a problem hiding this comment.
Same issue: SpanAttributes does not contain Pinecone-specific attributes.
Lines 131 and 138 use SpanAttributes.PINECONE_EMBEDDING_DIMENSIONALITY and SpanAttributes.PINECONE_RERANK_RESULT_COUNT, but these Pinecone-specific attributes should come from AISpanAttributes.
🐛 Proposed fix
if hasattr(first_emb, "values") and first_emb.values:
set_span_attribute(
span,
- SpanAttributes.PINECONE_EMBEDDING_DIMENSIONALITY,
+ AISpanAttributes.PINECONE_EMBEDDING_DIMENSIONALITY,
len(first_emb.values),
)
# For rerank responses
if hasattr(response, "results") and response.results:
set_span_attribute(
- span, SpanAttributes.PINECONE_RERANK_RESULT_COUNT, len(response.results)
+ span, AISpanAttributes.PINECONE_RERANK_RESULT_COUNT, len(response.results)
)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/opentelemetry-instrumentation-pinecone/opentelemetry/instrumentation/pinecone/__init__.py`
around lines 129 - 139, The Pinecone-specific span attributes used in the
set_span_attribute calls are incorrectly sourced from SpanAttributes instead of
AISpanAttributes. Replace SpanAttributes.PINECONE_EMBEDDING_DIMENSIONALITY with
AISpanAttributes.PINECONE_EMBEDDING_DIMENSIONALITY in the first
set_span_attribute call, and replace SpanAttributes.PINECONE_RERANK_RESULT_COUNT
with AISpanAttributes.PINECONE_RERANK_RESULT_COUNT in the second
set_span_attribute call. This ensures Pinecone-specific attributes are correctly
imported and referenced from the appropriate attributes class.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds OpenTelemetry tracing for Pinecone's Inference API calls:
New span attributes
Design decisions
Files changed
Fixes #1618
Summary by CodeRabbit
New Features
Version