perf(core): pipeline ancestor discovery

[jif-oai]

Why

Remote project startup still serialized most of its ancestor discovery:

• root markers only had an eight-request lookahead;
• AGENTS candidates and .agents/skills directories were checked one directory at a time;
• AGENTS and repository-skill discovery ran one after the other.

Each check is a normal fs/getMetadata RPC, so deep projects paid for hundreds of avoidable RTTs even though the paths are known from the cwd.

What changed

• Raise bounded lexical ancestor probing to 256 in-flight metadata calls.
• Probe AGENTS candidates concurrently across directories while preserving per-directory filename precedence and ordered error handling.
• Probe repository-skill roots and their project markers with the same bounded concurrency.
• Overlap independent AGENTS and plugin/skill discovery during session startup.

This does not add a filesystem batch API or change the wire protocol. Results are still consumed in the original deterministic order.

RTT benchmark

Nested AGENTS.md files	Before	After	Reduction
0	6.3 RTTs	4.2 RTTs	33.0%
12	70.3 RTTs	16.6 RTTs	76.4%
32	180.7 RTTs	38.3 RTTs	78.8%
64	315.4 RTTs	40.6 RTTs	87.1%

The depth-64 case removes 274.8 serialized RTT equivalents, a 7.8x improvement. The remaining plateau is mostly the existing 32 KiB AGENTS budget being filled by serial file reads; this PR keeps that separate from metadata discovery.

Measured with optimized app-server binaries, unique cold project fixtures, and a warm process/model/executor transport. Each point is the median of three campaigns with five samples at both 6 ms and 30 ms full RTT; the RTT count is derived from the wall-time delta.

Tests

• just test -p codex-core -E 'test(agents_md)' (57 passed)
• just test -p codex-core-skills (119 passed)
• repeated delayed-executor app-server benchmark (3 campaigns before and after, 15 samples per delay/point)