Add hot backup and restore support

[kdletters]

Description of Changes

Adds durable hot backup creation for running, disk-backed databases and offline restore support.

• Adds spacetime backup create and spacetime backup restore, plus a server endpoint confined to a configured hot-backup root.
• Captures an fsynced snapshot, waits for the matching durable transaction offset, copies the required commitlog range and scoped server state, and writes manifest.json last.
• Adds scheduled standalone backups under a per-database identity path, with ownership markers and scoped retention.
• Serializes backup against publish, reset, and delete operations, including cancellation-safe blocking finalization.
• Claims backup output paths across processes and rejects unsafe or overlapping destinations.
• Validates snapshot, commitlog, control-db, program bytes, metadata, and log-format compatibility before restore mutation.
• Aggregates rollback failures with recoverable paths and uses replace-existing atomic writes on Windows.
• Documents standalone configuration, scheduling, retention, and offline restore operation.

No related public ticket or PRD was identified.

AI Assistance

This change was developed with assistance from OpenAI Codex and GPT-5.5.

API and ABI breaking changes

This expands public Rust API surface with hot-backup manifest/in-progress types, default NodeDelegate methods, a provided SnapshotRepo invalidation helper, and Action::CreateHotBackup.

Existing trait implementations remain compatible because the new trait methods have defaults. Exhaustive downstream matches on Action may require an update for the new variant. No wire-format or binary ABI break is intended.

Expected complexity level and risk

5 / 5.

The change crosses snapshot creation, durability tracking, commitlog copying, control-db export, program storage, HTTP authorization, standalone lifecycle operations, filesystem crash consistency, restore rollback, scheduled retention, and Windows replacement semantics. The highest-risk boundaries are ensuring the snapshot never outruns durable history, keeping exported server metadata consistent with the replica, preserving the original target on failed restore, and preventing concurrent writers from sharing an output tree.

Testing

• cargo test -p spacetimedb-cli backup_restore --lib (30 passed)
• cargo test -p spacetimedb-engine --lib (55 passed, 1 ignored)
• cargo test -p spacetimedb-snapshot --lib (6 passed)
• cargo test -p spacetimedb-core hot_backup --lib (11 passed)
• cargo test -p spacetimedb-standalone --lib (21 passed)
• cargo test -p spacetimedb-client-api --lib (41 passed)
• cargo test -p spacetimedb-fs-utils (16 passed)
• Rust 1.93 clippy for the seven affected packages with --lib --tests -- -D warnings
• cargo fmt --all -- --check
• git diff --check upstream/master...HEAD
• Regenerated CLI docs; the candidate diff remained unchanged

[CLAassistant]

All committers have signed the CLA.