feat(spanner): support user-provided OpenTelemetry for client metrics export#13741
feat(spanner): support user-provided OpenTelemetry for client metrics export#13741rahul2393 wants to merge 1 commit into
Conversation
af8fd4a to
818893b
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces support for exporting Cloud Spanner client metrics to a caller-owned OpenTelemetry destination using a new MetricsProvider API, allowing users to configure custom pipelines independently of the default Google Cloud Monitoring export. It also updates documentation, adds comprehensive integration tests, and integrates routed endpoint attributes for location-aware routing. The review feedback highlights a bug in the newly added isEmulatorEnabled() method, where a mismatched protocol prefix (e.g., http://) between the configured host and the emulator host can cause the emulator detection to incorrectly return false.
| public boolean isEmulatorEnabled() { | ||
| return getChannelProvider() == null | ||
| && emulatorHost != null | ||
| && getHost() != null | ||
| && getHost().equals(emulatorHost); | ||
| } |
There was a problem hiding this comment.
The isEmulatorEnabled() method checks getHost().equals(emulatorHost). However, when setEmulatorHost is called with a value like "localhost:1234", the builder prepends "http://" to this.host (making it "http://localhost:1234"), but builder.emulatorHost remains "localhost:1234". This causes getHost().equals(emulatorHost) to return false because of the mismatched "http://" prefix.
To make this robust and handle both cases (with or without the protocol prefix), consider normalizing the comparison or checking if getHost() ends with emulatorHost or starts with it after stripping the protocol. Note that a null check on getHost() is unnecessary here as the host would have already been validated.
@InternalApi
public boolean isEmulatorEnabled() {
if (getChannelProvider() != null || emulatorHost == null) {
return false;
}
String normalizedHost = getHost().startsWith("http://") ? getHost().substring(7) : getHost();
String normalizedEmulatorHost = emulatorHost.startsWith("http://") ? emulatorHost.substring(7) : emulatorHost;
return normalizedHost.equals(normalizedEmulatorHost);
}References
- A null check on a parsed URL component (like the host) is unnecessary if other parts of the constructor would have already failed with a malformed URL (e.g., one missing a scheme).
… export Adds the MetricsProvider family (Default/Noop/CustomOpenTelemetry), SpannerOptions.Builder#setClientMetricsProvider, and SpannerMetrics#configureMeterProviderBuilder so client metrics can be recorded on a caller-owned OpenTelemetry in the spanner/client namespace, decoupled from the Cloud Monitoring built-in metrics flag. Also broadens the emulator gate (SpannerOptions#isEmulatorEnabled, enablegRPCMetrics overload) so no client metrics are recorded against the emulator. The emulator gate normalizes the http(s) scheme away before comparing the configured host with the emulator host.
818893b to
0d0ab92
Compare
Summary
Adds Client Metrics: a new, opt-in feature that exports Spanner's client-side metrics to a caller-provided OpenTelemetry pipeline (OTLP, Prometheus, any exporter), including on Spanner Omni where the existing Cloud Monitoring metrics are unavailable.
Fully opt-in and fully decoupled from the existing built-in (Cloud Monitoring) metrics. Default behavior is unchanged: without a client-metrics provider, nothing new happens, and built-in metrics continue to export to Cloud Monitoring exactly as before.
Two independent features
setBuiltInMetricsEnabledand theSPANNER_DISABLE_BUILTIN_METRICSenvironment variable, exactly as today. Not available on Spanner Omni.setClientMetricsProvider(...)— aCustomOpenTelemetryMetricsProviderturns it on;NoopMetricsProvider(or no provider) turns it off. Works on all instance types, Omni included.The two are decoupled:
setBuiltInMetricsEnabledand the env var affect only the built-in Cloud Monitoring sink and have no effect on client metrics; the client-metrics provider affects only the caller-owned sink and has no effect on built-in metrics. Under the hood these are the same client instruments, exported under the distinctspanner/clientnamespace — the difference is the export path, not the metrics.Emulator handling
Client metrics are not recorded when Spanner is pointed at the emulator. To make that reliable when the emulator is configured programmatically (via
setEmulatorHost(...)) and not only through theSPANNER_EMULATOR_HOSTenvironment variable, this PR broadens emulator detection:SpannerOptions.isEmulatorEnabled()now also recognizes the builder-configured emulator host, and the connection-check error message is generalized to describe both the environment-variable and programmatic configuration paths. This gates both the client and Cloud Monitoring metrics sinks off against the emulator using a single detection predicate. Called out explicitly here for reviewer visibility.Motivation
Client metrics currently export only to Cloud Monitoring, which is unavailable on Spanner Omni. Customers running on Omni (or who standardize on their own observability stack) had no way to receive these metrics. This lets them route the metrics to any OpenTelemetry exporter, independently of the built-in Cloud Monitoring configuration.
API