Add index discovery mode to support online index creation

[richhankins]

Overview

This PR adds dynamic index discovery to the context-connectors MCP server, enabling agents to see new indexes as they're added by users without restarting the MCP server.

Two Modes

Mode	CLI	Description
Fixed	ctxc mcp stdio -i pytorch -i react	Static index list from CLI args
Discovery	ctxc mcp stdio --discovery	Dynamic index list from filesystem

Tool Availability

Tool	Fixed Mode	Discovery Mode
search	✅ with enum	✅ no enum
list_files	✅ with enum	✅ no enum
read_file	✅ with enum	✅ no enum
list_indexes	✅ (returns static list)	✅ (required for discovery)
index_repo	❌	❌
delete_index	❌	❌

Note: In Discovery mode, users manage indexes via CLI (ctxc index ... commands), not via MCP tools.

Key Changes

1. Dynamic MultiIndexRunner

• Removed readonly from indexNames and indexes
• Added refreshIndexList() method to reload from store
• Added invalidateClient(name) to clear cached SearchClient
• Allowed empty index list on startup

2. list_indexes Tool

• New MCP tool to list available indexes with metadata
• Available in both Fixed and Discovery modes
• Returns index name, source, and last sync time

3. Mode-Based Tool Descriptions

• Fixed mode: Includes enum in tool schemas for zero-tool-call discovery
• Discovery mode: No enum, agent must call list_indexes
• Prevents stale enum values in discovery mode

4. CLI Flag

• Added --discovery flag for explicit mode selection
• Default (no flags) uses FilesystemStore in discovery mode
• -i flags use CompositeStoreReader in fixed mode

Benefits

1. User controls lifecycle - No accidental index creation/deletion by agents
2. No concurrency issues - Only user writes to the store via CLI
3. Simpler implementation - No write operations in MCP server
4. Incremental improvement - Minimal changes to existing MCP configs
5. Clear separation - Agent discovers, user manages

Example Workflow

Agent: "I need React Query docs to answer this question"
Agent: → list_indexes() → Not found
Agent: "I don't have React Query indexed. Would you like me to help you index it?"

User: "Yes"
User: $ ctxc index github TanStack/query

Agent: → list_indexes() → Now sees "tanstack-query"
Agent: → search("tanstack-query", "...")

Testing

• ✅ All 163 tests pass
• ✅ TypeScript compiles cleanly
• ✅ Manual testing via CLI and MCP tools
• ✅ Verified both Fixed and Discovery modes

Future Work

Discovery mode is an incremental step. A future PR can add full agent-managed mode with:

• index_repo and delete_index tools
• Writable FilesystemStore
• Optional LayeredStore for remote + local indexes
• Concurrency handling (lock files)