Add lazy segment loading for OFFLINE tables

[hash-14]

What

Adds opt-in lazy segment loading for OFFLINE tables: servers register a
metadata-only stub on ONLINE assignment instead of downloading the segment;
the first query that touches the segment materializes it from the deep store,
and a background sweeper evicts segments idle longer than a configurable TTL
back to stubs. Disk/memory then scale with the working set instead of total
assigned data. Entirely server-side — brokers, controllers and clients are
unchanged, and stubbed segments remain ONLINE and routable.

Resolves #18986

Why

Servers currently must hold 100% of every hosted table locally at all times,
so fleets are sized by total data volume rather than working set. Workloads
with a long cold tail (historical/audit data, sparse per-tenant tables) pay
for disk and memory they rarely use. See the linked issue for the full design
discussion.

How

• pinot-spi: new LazyLoadConfig (enabled, idleEvictionSeconds,
  deleteLocalOnEvict) on TableConfig; InstanceDataManagerConfig gains
  default methods for the three instance-level keys.
• pinot-common: ZK ser/de for lazyLoadConfig; new metrics
  LAZY_SEGMENT_COLD_LOADS, LAZY_SEGMENT_EVICTIONS,
  LAZY_SEGMENT_LOAD_TIME_MS, LAZY_STUBBED_SEGMENT_COUNT.
• pinot-segment-local: SegmentDataManager.lastAccessTimeMs stamped on
  acquire (idle clock); table-config validation restricting the feature to
  plain OFFLINE tables (rejected for REALTIME/upsert/dedup/dimension).
• pinot-core (BaseTableDataManager): stub registry; transparent
  materialization on acquireSegment(s) (required and optional segments)
  under the per-segment lock (concurrent cold queries dedupe into one
  download; multi-segment queries materialize in parallel on a bounded,
  idle-timeout pool, waiting at most
  lazy.materialize.timeout.seconds — still-stubbed segments are reported
  missing for that query while downloads continue in the background); acquire
  retries the eviction-sweeper race so no false missing-segment errors; stubs
  are registered only when the deep store has an authoritative copy and no
  valid local copy exists (restart warms up from disk; permanent deep-store
  failures still surface as Helix ERROR states); TTL eviction that never
  touches segments with in-flight queries or without a deep-store URL; failed
  materialization keeps the stub so the next query retries.
• pinot-server: pinot.server.instance.lazy.* config keys; single
  lazy-eviction-sweeper daemon in HelixInstanceDataManager.

Feature is off by default and opt-in twice: instance-level
pinot.server.instance.lazy.load.enabled=true AND per-table
lazyLoadConfig.enabled=true. Tables without lazyLoadConfig follow the
existing eager path exactly (covered by a regression test).

Testing

New OfflineTableDataManagerLazyLoadTest (TestNG, follows the
BaseTableDataManagerTest pattern with a file:// deep store):

• stub on assignment + materialization on both acquire paths
• 8-thread concurrent cold acquire dedupes into a single download
• TTL eviction and re-materialization loop
• no eviction while a query is in flight
• eager load when a valid local copy exists (restart warm-up)
• instance kill switch forces eager loading
• non-lazy table regression (behavior identical to upstream)

This feature has also been running in production since June 2026 (Pinot 1.5.0
base, MinIO deep store, including a 10 GB / 135M-row scale test).

Known limitations

Called out in the linked issue for design discussion (happy to address in
this PR or fast-follows per reviewer preference): stub sizes are not yet
included in table-size/quota reporting; materialization holds the per-segment
lock (slow downloads can delay Helix transitions on that segment); non-query
REST callers of acquireSegment materialize stubs as a side effect; reload
paths skip stubbed segments.

[hash-14]

cc @Jackie-Jiang @J-HowHuang @xiangfu0 — tagging you as recent committers to BaseTableDataManager / server segment management. This PR adds opt-in lazy segment loading for OFFLINE tables (design + known limitations in #18986, with production experience running it on Pinot 1.5.0). Would appreciate a review when you get a chance — happy to iterate on the design points raised in the issue.