Skip to content

MCP spec conformance: 4 requirement(s) violated (via @hasmcp/mcp-spec-test) — spec 2025-11-25 #248

Description

@hasmcp-dev

Running the black-box @hasmcp/mcp-spec-test conformance suite against opfor's MCP server (built from source: npm install && npm run build --workspaces --if-present, then node runners/mcp/dist/index.js) against the older 2025-11-25 revision it also advertises support for shows the same root cause as the newer revision (see the companion report filed alongside this one), with a smaller downstream failure count: the server prints a plaintext startup banner to stdout — ◇ injected env (0) from .env // tip: ⌘ enable debugging { debug: true } — before any JSON-RPC traffic, corrupting the stdio framing the MCP client relies on, so the process is torn down before the handshake completes and 13 further checks can't be verified. This is a real framing bug, not a test-environment artifact: on stdio transport, an MCP server must write only JSON-RPC to stdout.

Conformance report

MCP 2025-11-25 conformance report

Verdict: not conformant — 4 requirements violated.

Target sh -c '[ -d agent-opfor ] || git clone https://github.com/KeyValueSoftwareSystems/agent-opfor && cd agent-opfor && npm install && npm run build --workspaces --if-present && node runners/mcp/dist/index.js'
Transport stdio
Revision tested 2025-11-25
Revisions supported 2026-07-28, 2025-11-25
Passed 3
Failed 4
Not verified 13
Cases applied 20
Duration 19290ms
Generated 2026-08-24T14:18:49.031Z
Tool @hasmcp/mcp-spec-test 0.1.1

Failed (4)

the server deviates from the spec here

Version negotiation

  • a request with no version at all is served on the default
    a version-less request must be served, got 0: {"_raw":"server exited"}

    0 !== 200

  • an unsupported version offered at the handshake is refused or downgraded, not echoed
    expected either an error or a protocolVersion, got {"_raw":"server exited"}

Official SDK interop

  • a stock official-SDK client completes the handshake
    MCP error -32000: Connection closed
  • the handshake settles on a revision inside the supported window
    initialize must return a protocolVersion, got undefined

Not verified (13)

skipped; a skip is not a pass

Capability methods

  • tools/list returns schema-conformant tools
    cannot determine what the target supports — initialize returned no result (status 0)
  • tools/call on an unknown tool is an error, not a crash
    cannot determine what the target supports — initialize returned no result (status 0)
  • tools/call returns a schema-conformant CallToolResult
    cannot determine what the target supports — initialize returned no result (status 0)
  • prompts/list returns schema-conformant prompts
    cannot determine what the target supports — initialize returned no result (status 0)
  • prompts/get returns messages with a role and content
    cannot determine what the target supports — initialize returned no result (status 0)
  • resources/list returns schema-conformant resources
    cannot determine what the target supports — initialize returned no result (status 0)
  • resources/templates/list returns schema-conformant templates
    cannot determine what the target supports — initialize returned no result (status 0)
  • resources/read returns contents for every sampled resource
    cannot determine what the target supports — initialize returned no result (status 0)
  • resources/read on an unknown uri is an error
    cannot determine what the target supports — initialize returned no result (status 0)
  • following nextCursor terminates and does not repeat a page
    cannot determine what the target supports — initialize returned no result (status 0)
  • an invalid pagination cursor is rejected (SHOULD)
    cannot determine what the target supports — initialize returned no result (status 0)

Version negotiation

  • initialize returns the schema-required fields
    no session — initialize returned no result (status 0)
  • the handshake settles on the revision under test
    no session — initialize returned no result (status 0)

Passed (3)

checked and conformant

server/discover

  • the suite is reading a schema that matches the features it selected

Official SDK interop

  • the official SDK does not yet implement the newest revision
  • a stock official-SDK client can list tools

How to reproduce

With Docker:

docker pull hasmcp/mcp-spec-test:latest
docker run --rm hasmcp/mcp-spec-test -c "sh -c '[ -d agent-opfor ] || git clone https://github.com/KeyValueSoftwareSystems/agent-opfor && cd agent-opfor && npm install && npm run build --workspaces --if-present && node runners/mcp/dist/index.js'" --spec-version 2025-11-25

(on Apple Silicon / arm64 hosts, add --platform linux/amd64; the published image currently only ships an amd64 manifest but runs fine under emulation)

Without Docker (npx):

npx @hasmcp/mcp-spec-test@latest -c "sh -c '[ -d agent-opfor ] || git clone https://github.com/KeyValueSoftwareSystems/agent-opfor && cd agent-opfor && npm install && npm run build --workspaces --if-present && node runners/mcp/dist/index.js'" --spec-version 2025-11-25

You can see the offending stdout line directly with:

node runners/mcp/dist/index.js
# stdout prints: ◇ injected env (0) from .env // tip: ⌘ enable debugging { debug: true }

This report was generated by an automated MCP conformance sweep across public MCP server repos. If this turns out to be a false positive specific to the test container/environment, or not applicable, feel free to close it. (A companion report against the 2026-07-28 revision this server also supports, with a larger failure set, is filed separately.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions