feat: expose summary metrics through the mcp

[karl-power]

Summary metrics are stored in ClickHouse and declared on metric sources, but renderChartConfig cannot translate them — so the MCP hid them entirely: clickstack_list_metrics skipped the kind, clickstack_describe_metric rejected it, and an agent asking about them hit a dead end with no guidance. Agents could not even learn these metrics existed.

This PR makes summary metrics discoverable through the MCP while keeping them un-queryable by the builder tools: every surface that lists or describes them directs agents to clickstack_sql instead. No renderer changes; no SQL recipes baked into tool output.

What changed

MCP discovery (packages/api/src/mcp/)

• New DISCOVERABLE_METRIC_KINDS (= queryable kinds + summary) feeding clickstack_list_metrics (scan, kind filter, cursor codec), clickstack_describe_source name sampling, and clickstack_describe_metric (kind:"summary" now returns attribute keys/values/unit/description, with a clickstack_sql next-step instead of a timeseries example)
• clickstack_timeseries / clickstack_table / dashboard tile schemas still reject summary; their descriptions, plus list_sources, clickstack_sql, and the dashboards prompt, now carry a one-line redirect: query the table in metricTables.summary via clickstack_sql

Eval verification (packages/hdx-eval/)

• metric-saturation redesigned so the summary metric is load-bearing: misleading "Feature-store query timed out" WARN logs (and the reporter's prompt) blame the shared product_catalog Postgres; the only exoneration is inventory-service's flat db.client.operation.duration summary, readable only via raw SQL
• New rubric checks: discovered_summary_metric / queried_summary_via_sql (adoption), exonerates_database (positive), and a negation guard on false_blame_database so "not the database" doesn't trip it

Results

12-run batch (metric-saturation, claude-fable-5 + claude-opus-4-6, 3 runs/cell). The hyperdx arm has this PR's discovery + redirect; the clickhouse arm must find the summary table on its own.

Check	hyperdx MCP	clickhouse MCP
Discovered the summary metric	6/6	1/6
Queried its quantiles via raw SQL	6/6	1/6
Exonerated the database	6/6	4/6
Falsely blamed the database	0/6	0/6

Every hyperdx-arm run followed the full path — discovery → redirect → clickstack_sql against the summary table — and reached the correct root cause ("JVM heap saturation, not the database"). Before this PR the same checks read 0/6 by construction: the metric name was invisible to agents.

References

• Linear Issue: Closes HDX-4767

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperdx-oss	Ready	Preview	Aug 10, 2026 4:51pm
hyperdx-storybook	Ready	Preview	Aug 10, 2026 4:51pm

[changeset-bot]

🦋 Changeset detected

Latest commit: c9419b0

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages

Name	Type
@hyperdx/hdx-eval	Patch
@hyperdx/api	Patch
@hyperdx/app	Patch
@hyperdx/otel-collector	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

🔵 Tier 2 — Low Risk

Small, isolated change with no API route or data model modifications.

Why this tier:

• Standard feature/fix — introduces new logic or modifies core functionality

Additional context: 2 file(s) in private internal-tooling packages, excluded from the line count

Review process: AI review + quick human skim (target: 5–15 min). Reviewer validates AI assessment and checks for domain-specific concerns.
SLA: Resolve within 4 business hours.

Stats

• Production files changed: 12
• Production lines changed: 174 (+ 184 in test files, excluded from tier calculation)
• Branch: karl/mcp-expose-summary-metrics
• Author: karl-power

To override this classification, remove the review/tier-2 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

[greptile-apps]

Greptile Summary

The PR exposes summary metrics through MCP discovery while keeping builder tools restricted to renderer-supported metric kinds.

• Adds summary-aware metric listing, description, cursor handling, and raw-SQL guidance.
• Updates MCP schemas and dashboard prompts to redirect summary queries to clickstack_sql.
• Revises the metric-saturation evaluation scenario and grading to exercise summary discovery and SQL querying.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
packages/api/src/mcp/tools/sources/describeMetric.ts	Adds summary discovery and a database-qualified raw-SQL redirect.
packages/api/src/mcp/tools/sources/metricKinds.ts	Separates discoverable metric kinds from kinds supported by builder queries.
packages/api/src/mcp/tools/sources/listMetrics.ts	Includes summary metrics in catalog scans, filters, and pagination.
packages/hdx-eval/src/scenarios/metric-saturation/generate.ts	Makes the summary metric and misleading database-timeout evidence load-bearing in the evaluation.
packages/hdx-eval/src/scenarios/metric-saturation/ground-truth.json	Adds summary adoption checks and calibrated database-exoneration grading.

_{Reviews (4): Last reviewed commit: "review feedback" | Re-trigger Greptile}

[github-actions]

E2E Test Results

✅ All tests passed • 276 passed • 1 skipped • 1098s

Status	Count
✅ Passed	276
❌ Failed	0
⚠️ Flaky	1
⏭️ Skipped	1

Tests ran across 4 shards in parallel.

View full report →

[github-actions]

Deep Review

Scope: git diff 8351d632 — 20 files, MCP summary-metric discovery in packages/api/src/mcp/ plus eval scaffolding in packages/hdx-eval/. Mode: report-only (no edits made).

Intent (advisory): make OTel summary metrics discoverable through the MCP discovery tools while keeping them un-queryable by the builder tools, redirecting agents to clickstack_sql.

Core invariant verified directly: the builder/dashboard schemas still bind metricType to z.enum(QUERYABLE_METRIC_KINDS) (packages/api/src/mcp/tools/query/schemas.ts:82), so summary is rejected at Zod validation — not merely documented as unsupported. Only the discovery-tool enums (describeMetric.kind, listMetrics.kind) and the cursor codec widen to DISCOVERABLE_METRIC_KINDS. The split is applied consistently, and pickRepresentativeMetricTable still ranges over queryable kinds only, so a summary table can never become the schema-discovery representative. The performance reviewer confirmed the discovery fan-out stays a fixed-width (4→5), gated, signal-aware, timeout-capped set of queries. This is a clean, well-tested change with no critical issues.

✅ No critical issues found.

🟡 P2 -- recommended

• packages/api/src/mcp/tools/query/schemas.ts:82 -- The feature's load-bearing safety invariant (builder tools reject summary) has no explicit negative test; the int test only asserts describe/list accept summary and that the redirect text avoids a clickstack_timeseries({ example, so a future widening of the query enum to DISCOVERABLE_METRIC_KINDS would pass CI while silently reaching the renderer that throws on summary.
  • Fix: Add a test that submits metricType:"summary" to clickstack_timeseries / clickstack_table and asserts a validation error is returned.

🔵 P3 nitpicks (2)

🔵 P3 nitpicks (2)

• packages/api/src/mcp/tools/sources/describeSource.ts:55 -- For a metric source whose metricTables contains only a summary table, pickRepresentativeMetricTable returns undefined, leaving top-level column/attribute discovery empty even though summary metric names are still sampled; this is acceptable (summary is not chartable) but is an undocumented discovery edge.
  • Fix: Add a brief comment noting summary-only sources yield name samples without representative schema discovery, or confirm the existing behavior is intended.
• packages/api/src/mcp/tools/sources/listMetrics.ts:987 -- The "summary metrics cannot be queried … use clickstack_sql against the table in the source's metricTables.summary" sentence is duplicated verbatim across ~7 tool descriptions/prompts and can drift out of sync as guidance evolves.
  • Fix: Optionally hoist the redirect sentence into a shared constant reused by the discovery/builder descriptions.

---

Reviewers (10 dispatched): correctness, adversarial, testing, maintainability, project-standards, api-contract, performance, kieran-typescript, agent-native, learnings-researcher. Note: at synthesis time only performance had returned (no findings); the P2/P3 items above were derived from direct diff analysis by the orchestrator. Re-run for the full panel's corroboration if desired.

Testing gaps:

• No negative test that builder tools reject metricType:"summary" — the guarantee that makes the feature safe is untested.
• Summary-only metric source (no queryable kinds populated) discovery path is not exercised by a test.

[pulpdrew]

LGTM, with a question

[pulpdrew]

Just in case you haven't seen it, there is also this PR that is intended to discourage agents from using raw SQL: #2840

Does adding this here (or the clickstack_sql elsewhere) have any effect on that goal / do we need to run any evals to measure the effect? cc @brandon-pereira

[pulpdrew]

I'm also curious, does the agent actually generate reasonably accurate queries for summary metrics? The metrics queries tend to be very complicated.