store: merge adjacent preads in BucketStorageBackend::BatchLoad

[pingzhuu]

Sort keys by file offset within each bucket, then merge groups of keys whose data ranges are within 256KiB of each other into a single pread call. This reduces syscall count and per-call gap overhead by ~50% (measured on 8-disk NVMe cluster with 64K-token prompts).

Also adds VLOG(1) diagnostic logging in BatchLoad and batch_get_into for debugging SSD read patterns.

Set CROSSING_LOAD_CHUNK_TOKENS=131072 in prefill template to batch the entire prompt into a single store RPC.

Description

Module

• Transfer Engine (mooncake-transfer-engine)
• Mooncake Store (mooncake-store)
• Mooncake EP (mooncake-ep)
• Mooncake PG (mooncake-pg)
• Integration (mooncake-integration)
• P2P Store (mooncake-p2p-store)
• Python Wheel (mooncake-wheel)
• Common (mooncake-common)
• Mooncake RL (mooncake-rl)
• CI/CD
• Docs
• Other

Type of Change

• Bug fix
• New feature
• Refactor
• Breaking change
• Documentation update
• Performance improvement
• Other

How Has This Been Tested?

Test commands:

# Example: bash scripts/run_ci_test.sh

Test results:

• Unit tests pass
• Integration tests pass (if applicable)
• Manual testing done (describe below)

Checklist

• I have performed a self-review of my own code
• I have formatted my code using ./scripts/code_format.sh
• I have run pre-commit run --all-files and all hooks pass
• I have updated the documentation (if applicable)
• I have added tests to prove my changes are effective
• For changes >500 LOC: I have filed an RFC issue

AI Assistance Disclosure

• No AI tools were used
• AI tools were used (specify below)