Skip to content

fix(spanner): derive built-in metrics project from database client#13262

Open
rahul2393 wants to merge 2 commits into
mainfrom
fix-builtin-metrics-spanner
Open

fix(spanner): derive built-in metrics project from database client#13262
rahul2393 wants to merge 2 commits into
mainfrom
fix-builtin-metrics-spanner

Conversation

@rahul2393
Copy link
Copy Markdown
Contributor

@rahul2393 rahul2393 commented May 25, 2026

Fixes #13240

Fix built-in metrics export project selection for shared VPC / host-project environments.

Built-in metrics can initialize before any DatabaseClient, causing the Cloud Monitoring exporter to use
SpannerOptions.getProjectId() instead of the Spanner database project. This makes createServiceTimeSeries target the wrong project.

Changes:

  • Set built-in metrics project from the first DatabaseId used by getDatabaseClient() or getBatchClient()
  • Keep first database project as the exporter project
  • No-op metrics export until a database/batch client initializes the project
  • Override monitored resource project_id during export because OTel Resource is immutable

@rahul2393 rahul2393 requested review from a team as code owners May 25, 2026 09:20
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the built-in metrics initialization and export logic to use a project ID supplier, ensuring that metrics are correctly associated with the active database project. Key changes include updating SpannerCloudMonitoringExporter to override resource project IDs during export and integrating metric initialization into SpannerImpl and SpannerOptions. Reviewer feedback identifies dead code in the exporter, suggests optimizing convertToSpannerTimeSeries to accept a Collection to avoid redundant list copies, and points out a test case that needs a non-empty collection to properly validate export skipping behavior.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the initialization and export of built-in metrics in the Spanner client by introducing a project ID supplier and triggering initialization upon client access. The logic for filtering metrics based on project ID mismatches in resource attributes has been removed, with the project ID now applied directly to all exported metrics. Review feedback suggests enhancing performance by using Collection instead of List in convertToSpannerTimeSeries to avoid redundant object allocations. Furthermore, the reviewer recommended re-evaluating the lastExportSkippedData flag, as the removal of filtering logic has rendered its current implementation potentially obsolete or incorrect.

BuiltInMetricsView.registerBuiltinMetrics(
SpannerCloudMonitoringExporter.create(
projectId, credentials, monitoringHost, universeDomain),
this::getProjectId, credentials, monitoringHost, universeDomain),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rahul2393 I did not understand this solution. getOrCreateOpenTelemetry is called from GapicSpannerRPC while creating SpannerClient. At the time projectId shared here could be the projectId of GKE instance for example.

So in this case we will be initialising SpannerCloudMonitoringExporter with null projectId ? As by this time setProjectIdIfAbsent won't be called, it is called later during database init.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So flow is:

  1. SpannerClient init → OpenTelemetry/exporter may be created, project supplier returns null
  2. no export happens yet because no database project is known
  3. getDatabaseClient(DatabaseId) → database project is set once
  4. future metric exports use that database project

BuiltInMetricsView.registerBuiltinMetrics(
SpannerCloudMonitoringExporter.create(
projectId, credentials, monitoringHost, universeDomain),
this::getProjectId, credentials, monitoringHost, universeDomain),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are also passing the projectId in next line to create OpenTelemetry Resource

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also do

monitoredResourceBuilder.putLabels(PROJECT_ID_KEY.getKey(), projectId);

So the resource created during SDK initialization may contain the early/default project, but before sending
createServiceTimeSeries, we overwrite the monitored resource label with the database project from the exporter supplier.

@rahul2393 rahul2393 requested a review from surbhigarg92 May 25, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spanner: client-side metrics export uses wrong project

2 participants