Skip to content

Add cache_hints constructor map for SEP-2549 caching hints#3015

Draft
maxisbey wants to merge 1 commit into
mainfrom
cache-hints
Draft

Add cache_hints constructor map for SEP-2549 caching hints#3015
maxisbey wants to merge 1 commit into
mainfrom
cache-hints

Conversation

@maxisbey

Copy link
Copy Markdown
Contributor

Adds a cache_hints constructor option to Server and MCPServer so server operators can set the SEP-2549 caching hints (ttlMs / cacheScope) emitted on the six cacheable results.

Motivation and Context

The 2026-07-28 schema requires ttlMs/cacheScope on the results of tools/list, prompts/list, resources/list, resources/templates/list, resources/read, and server/discover. The models default them to 0/"private" (immediately stale, never shared) — wire-valid and safe, but until now there was no way for an MCPServer operator to declare anything else: every high-level server told every client "don't cache this", regardless of intent. Lowlevel handlers could set the fields on the results they return, but only per handler.

This adds the server-wide knob:

from mcp.server import CacheHint, MCPServer

mcp = MCPServer(
    "demo",
    cache_hints={
        "tools/list": CacheHint(ttl_ms=60_000, scope="public"),
        "resources/read": CacheHint(ttl_ms=5_000),  # scope stays "private"
    },
)

Design notes:

  • The map lives on the lowlevel Server (next to instructions=); MCPServer forwards it. One mechanism covers all six methods, including server/discover, whose default handler is lowlevel-owned.
  • ServerRunner fills the typed result after the handler returns and before serialization, so the existing version sieve still strips the fields for pre-2026 peers, and resultType: "input_required" results are never stamped (they aren't CacheableResult models).
  • Precedence is per field via model_fields_set: a field the handler set explicitly — even to the default value — always wins over the configured hint. A configured hint can never clobber a handler's explicit cache_scope.
  • Keys are typed as a CacheableMethod Literal (the analogue of the TypeScript SDK's Partial<Record<CacheableResultMethod, CacheHint>>), so editors autocomplete them and type checkers flag typos. Runtime validation still rejects unknown keys (ValueError) and non-CacheHint values (TypeError) at construction for untyped callers.
  • The same cacheScope reaches every page of a paginated list by construction: the map is keyed by method, not cursor.

How Has This Been Tested?

  • New tests in tests/server/test_caching.py: hint validation, per-field precedence (including explicit-default-wins), wire round-trips on both server tiers, discover, and per-page scope consistency.
  • A sieve test in tests/server/test_runner.py proving configured hints never reach a 2025-11-25 peer.
  • New docs page docs/advanced/caching.md with runnable docs_src/caching/ examples pinned by tests/docs_src/test_caching.py.
  • Full suite: 4258 passed, 100% branch coverage, pyright/ruff clean. The conformance caching scenario (CI) covers the emitted-hints requirements.

Breaking Changes

None — new optional keyword argument; defaults unchanged.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

Part of #2899. Client-side cache honoring (a freshness cache that uses these hints) and a per-registration @mcp.resource(..., cache_hint=...) override are intentionally out of scope for this PR.

AI Disclaimer

Server and MCPServer take cache_hints={method: CacheHint(...)} to set
ttlMs/cacheScope on the six cacheable results server-wide. The runner
fills the typed result after the handler returns, so fields a handler
sets explicitly win, per field (via model_fields_set), and the existing
serialize sieve keeps pre-2026 wires clean. Keys are typed as the
CacheableMethod Literal so editors autocomplete them and flag typos;
runtime validation still rejects bad keys and values at construction
for untyped callers.

Part of #2899.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant