Long-lived containers permanently retain stale secrets (<UNKNOWN>) after secret is stored

[henryfrog333-tech]

Bug Description

When a longLived: true MCP server is spawned before its secret is stored in the keychain, the container gets APIFY_TOKEN=<UNKNOWN> baked into its environment. This value persists permanently because long-lived containers are never recreated, even after the secret is correctly stored via docker mcp secret set.

Steps to Reproduce

1. Enable a server with longLived: true in the catalog (e.g., apify-mcp-server)
2. Do not set its secret yet
3. Invoke any tool from that server — the gateway spawns the container with SECRET_VAR=<UNKNOWN>
4. Now set the secret: echo 'valid-token' | docker mcp secret set apify-mcp-server.apify_token
5. Invoke the tool again — it still fails because the running container still has <UNKNOWN>

Expected Behavior

Either:

• (A) The gateway should refuse to start a container when a required secret resolves to <UNKNOWN>, instead of silently proceeding
• (B) The gateway should recreate long-lived containers when secrets change in the keychain
• (C) The --watch flag should monitor the secret store, not just config files

Actual Behavior

• The gateway logs Warning: Secret 'X' not found for server 'Y', setting Z=<UNKNOWN> but proceeds anyway
• The container starts and tools are listed successfully (tools/list works)
• Actual tool calls fail with cryptic upstream auth errors (e.g., "User was not found or authentication token is not valid")
• The container persists across sessions because longLived: true
• Even after the secret is stored, the running container is never refreshed

Proof

# Container running with stale <UNKNOWN>
$ docker inspect c37a7f65a635 --format '{{range .Config.Env}}{{println .}}{{end}}'
APIFY_TOKEN=<UNKNOWN>
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
NODE_VERSION=24.14.0

# Secret IS in the keychain
$ docker mcp secret ls | grep apify
docker/mcp/apify-mcp-server.apify_token | docker-pass

# Fresh gateway (via CLI) works fine — proves secret is stored correctly
$ echo '{"keywords":"test","limit":1}' | docker mcp tools call search-actors
# Returns valid results

Impact

• Affected servers: Any server with both longLived: true AND secrets defined in the catalog. Currently: apify-mcp-server, schemacrawler-ai
• User impact: Hours of debugging. The error appears to be an auth/token issue, but the root cause is a stale container that was created before the secret existed
• Workaround: docker kill <container_id> then restart the MCP client to get a fresh gateway connection

Environment

• Docker MCP Toolkit: v0.40.1
• Docker Desktop: macOS (Darwin 25.2.0)
• Credential store: docker-pass (via Docker Desktop)

Related Issues

• Bypass Docker MCP secret store; pass secrets from environment variables instead #317 — Gateway overwrites env vars / secret store bypass
• Env variables getting overwritten #140 — cmd.Env drops parent environment
• Secrets example doesnt work on server "Error: open /.s0: file does not exist" #75 — Secrets require Docker Desktop (jfs.sock)

Suggested Fix

In the gateway's container creation path, add a check:

// Before creating a long-lived container:
for _, secret := range server.Secrets {
    if resolvedValue == "<UNKNOWN>" {
        return fmt.Errorf("required secret %q not found for server %q; refusing to start container with placeholder value", secret.Name, server.Name)
    }
}

Additionally, consider watching the secret store for changes and triggering container recreation for affected long-lived servers.