Skip to content

perf(cloud): broadcast signals and storage-backed replay segments#511

Merged
Harry19081 merged 4 commits into
developfrom
codex/broadcast-and-storage-offload
Jul 24, 2026
Merged

perf(cloud): broadcast signals and storage-backed replay segments#511
Harry19081 merged 4 commits into
developfrom
codex/broadcast-and-storage-offload

Conversation

@Neonforge98

@Neonforge98 Neonforge98 commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

Client side of ORGII-cloud-infra 0005_broadcast_and_storage_offload.sql — the two platform-level scalability directions the audit left open (H4 + H5). Both are capability-gated via the new get_cloud_capabilities() probe (missing RPC ⇒ legacy behavior, remembered per endpoint), so the client is fully backward compatible with pre-0005 backends and old clients keep working against a 0005 backend.

H4 — change signals via Broadcast-from-Database. The client consumed zero row data from its postgres_changes subscriptions (pure invalidation edges), so signals now arrive as server-sent org-db-changed {kind} broadcasts on the already-open private org channel. On broadcast-capable backends the two org-scoped postgres_changes channels are not created (channels per client drop 4 → 2), removing the per-change × per-subscriber RLS evaluation on the shared WAL poller — the platform's first scalability wall. Missed-signal recovery (disconnect-gated full/delta) moves to the org channel's SUBSCRIBED edge; the self-roster eviction slice deliberately stays on postgres_changes (a removed member is unreachable by join-time-auth broadcast). roster broadcasts also run the coarse recovery because the server's per-org debounce makes a member-floor change broadcast as roster (kind shadowing, documented in the migration).

H5 — frozen replay segments offloaded to Supabase Storage. Frozen segments (seq ≥ 1, immutable bulk) upload as raw gzip objects ({org}/{session}/{epoch}/{seq}-{hash}.gz, drops the base64 +33%) before the manifest RPC commits them; the server verifies object existence/size/hash-embedded-name against storage.objects (server-measured quota, no client attestation). The mutable tail stays inline in Postgres, keeping append OCC single-transaction. Reads decode storagePath XOR payloadGz per segment; client integrity validation (sha256 over pre-gzip bytes) is unchanged.

Test plan

  • tsc clean; Org2Cloud + TeamCollaboration slices 905/905 (26 new tests: capability probe caching, broadcast event dispatch + presence-channel status edges, storage PUT/GET wire shapes, storage-form append/rewrite, PGRST202 fallback + endpoint memory, ORG2_VALIDATION no-fallback, mixed inline/storage page decode, fail-closed cases); full suite green at branch point.
  • Server counterpart validated twice on disposable Postgres 16 (author agent matrix + independent re-run): 0001→0005 sequence + idempotent re-run, inline-form byte-identical A/B, storage-form quota exactness, read XOR projection, storage RLS helper ladder (member/owner/retention/access-mode/restricted), orphan view, debounced broadcast smoke (1 write ⇒ 1 private-topic message, suppress window intact, ACL denials).
  • Real-machine dual-instance verification pending live apply (broadcast delivery, storage push/pull, cmd+5 + CPU/RAM).

Pre-commit hook ran. Total eslint: 1, total circular: 0
Pre-commit hook ran. Total eslint: 1, total circular: 0
Pre-commit hook ran. Total eslint: 1, total circular: 0
Pre-commit hook ran. Total eslint: 1, total circular: 0
@Harry19081
Harry19081 merged commit 93056a3 into develop Jul 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants