[AIT-1106] fix(uts): correct objects proxy specs and harden proxy rule/timeout guidance

uts/README.md

@@ -28,14 +28,13 @@ uts/
 │   │   ├── connection/                # RTN — connection management
 │   │   └── presence/                  # RTP — realtime presence
 │   └── integration/                   # Realtime integration tests
-│       ├── helpers/
-│       │   └── proxy.md               # Proxy infrastructure spec
 │       ├── proxy/                     # Proxy-based fault injection tests
 │       └── *.md                       # Direct sandbox tests
 ├── docs/                              # Guides and reference
 │   ├── writing-test-specs.md          # How to write UTS specs
 │   ├── writing-derived-tests.md       # How to translate specs into SDK tests
 │   ├── integration-testing.md         # Integration testing policy
+│   ├── proxy.md                       # Proxy infrastructure spec (cross-module)
 │   └── completion-status.md           # Spec coverage matrix
 └── README.md                          # This file
 ```
@@ -101,4 +100,4 @@ See [docs/writing-test-specs.md](docs/writing-test-specs.md) for the full pseudo
 
 The programmable proxy for integration testing lives in a separate repository: [ably/uts-proxy](https://github.com/ably/uts-proxy). It sits between the SDK and the Ably sandbox, transparently forwarding traffic while allowing rule-based fault injection.
 
-See `realtime/integration/helpers/proxy.md` for the proxy infrastructure specification used by test specs in this repository.
+See `uts/docs/proxy.md` for the proxy infrastructure specification used by test specs in this repository.

uts/docs/integration-testing.md

@@ -99,8 +99,6 @@ realtime/
     presence/
       presence_lifecycle_test.md
       ...
-    helpers/
-      proxy.md                             # Proxy infrastructure spec
     proxy/                                 # Proxy-based tests (sandbox + proxy)
       connection_open_failures.md
       connection_resume.md
@@ -151,7 +149,7 @@ AFTER ALL TESTS:
 
 ### Proxy Setup (integration-proxy only)
 
-Proxy tests additionally set up a proxy session per test or group of tests. See `realtime/integration/helpers/proxy.md` for the proxy infrastructure API.
+Proxy tests additionally set up a proxy session per test or group of tests. See `uts/docs/proxy.md` for the proxy infrastructure API.
 
 ```pseudo
 BEFORE EACH TEST:
@@ -227,6 +225,12 @@ Guidelines:
 
 The goal is: every await in the test is bounded, and the suite timeout is generous enough that it only fires if something truly unexpected happens. When a test fails, the error should say *what* timed out, not just "suite timeout exceeded."
 
+**All timeouts are wall-clock time.** Every `WITH timeout`, `poll_until` and `WAIT` in an
+integration spec measures real elapsed time — the test is waiting on a real server over a real
+network. Derived tests in frameworks that virtualise time by default must run these waits
+against the real clock; see the *Integration timeouts are wall-clock* section in
+`writing-derived-tests.md` for the failure mode and recommended helpers.
+
 ### Avoiding Flaky Tests
 
 - Use polling with timeouts instead of fixed waits (see `README.md` polling conventions)

uts/realtime/integration/helpers/proxy.md → uts/docs/proxy.md

@@ -112,6 +112,14 @@ Rules are evaluated in order. First matching rule wins. Unmatched traffic passes
 
 **`count`**: 1-based occurrence counter. `count: 2` matches only the 2nd occurrence.
 
+**`action`** (frame matches): must be a **string** — either a protocol action *name*
+(e.g. `"ATTACHED"`) or a *numeric string* (e.g. `"20"`). A JSON number is rejected at
+session creation with HTTP 400 (`cannot unmarshal number into Go struct field
+MatchConfig.match.action of type string`). Note: uts-proxy v0.3.0 resolves action
+*names* only up to `AUTH` (17) — an unresolvable name (e.g. `"OBJECT_SYNC"`) makes the
+rule silently never match, so use numeric strings for `OBJECT` (`"19"`),
+`OBJECT_SYNC` (`"20"`) and `ANNOTATION` (`"21"`) until the proxy's name table is extended.
+
 ### Actions
 
 ```json
@@ -233,3 +241,6 @@ ClientOptions(
 7. Timeouts are generous (10-30s) since real network is involved
 8. Each test file provisions a sandbox app in `BEFORE ALL TESTS` and cleans up in `AFTER ALL TESTS`
 9. Each test creates its own proxy session and cleans it up after
+10. All `WITH timeout` / `poll_until` / `WAIT` durations are **wall-clock (real) time** — see
+    the *Integration timeouts are wall-clock* section in `writing-derived-tests.md` for
+    the virtual-time trap in derived tests

uts/docs/writing-derived-tests.md

@@ -268,6 +268,27 @@ const timer = setTimeout(() => reject(new Error('Timed out')), 5000);
 connection.once('connected', () => { clearTimeout(timer); resolve(); });
 ```
 
+### Integration timeouts are wall-clock (beware virtual-time frameworks)
+
+The rule above is inverted for **integration and proxy tests**: every `WITH timeout`,
+`poll_until` and `WAIT` in an integration spec is **wall-clock (real) time**, because the test
+is waiting on a real server or proxy over a real network.
+
+This is a trap in test frameworks that virtualise time by default. For example,
+kotlinx-coroutines' `runTest` runs the test body on a virtual clock: a bare `withTimeout(15s)`
+wrapping a real network await measures *virtual* time, which fast-forwards the moment the test
+coroutine idles — the timeout fires almost instantly, long before the server can respond, with
+a misleading "Timed out after 15s" error. The same applies to a bare `delay()`, which skips
+instead of waiting.
+
+Derived integration tests in such frameworks must run their waits against the real clock —
+e.g. dispatch onto a real-thread dispatcher before applying the timeout
+(`withContext(Dispatchers.Default.limitedParallelism(1)) { withTimeout(...) { ... } }` in
+Kotlin), or use the framework's escape hatch for real time. Define shared helpers
+(`awaitState`, `pollUntil`, `withRealTimeout`, ...) that encapsulate this once, and use them for
+every wait in integration test bodies. Unit tests are unaffected — there, fake/virtual timers
+remain the preferred mechanism.
+
 ### Cleanup with afterEach
 
 Always restore mocks in `afterEach`, not just at the end of each test. If a test throws before its cleanup code, the next test inherits dirty state. Use the SDK's mock restoration mechanism (e.g. `restoreAll()`) in an `afterEach` hook.

uts/docs/writing-test-specs.md

@@ -21,7 +21,7 @@ This guide provides comprehensive guidance for writing portable test specificati
 - Run against Ably Sandbox through a programmable proxy ([ably/uts-proxy](https://github.com/ably/uts-proxy))
 - Proxy transparently forwards traffic but can inject faults via rules
 - Use for testing fault behaviour: connection failures, token renewal under errors, heartbeat starvation, channel error injection
-- See `realtime/integration/helpers/proxy.md` for the full proxy infrastructure spec
+- See `uts/docs/proxy.md` for the full proxy infrastructure spec
 
 ## Test IDs
 
@@ -296,7 +296,7 @@ mock_ws.active_connection.simulate_disconnect()
 
 ## Proxy Integration Tests
 
-For detailed proxy infrastructure documentation, see `realtime/integration/helpers/proxy.md`.
+For detailed proxy infrastructure documentation, see `uts/docs/proxy.md`.
 
 ### When to Use Proxy Tests
 
@@ -317,7 +317,7 @@ Spec points: `RTN14a`, `RTN14b`, ...
 Proxy integration test against Ably Sandbox endpoint
 
 ## Proxy Infrastructure
-See `realtime/integration/helpers/proxy.md` for proxy infrastructure specification.
+See `uts/docs/proxy.md` for proxy infrastructure specification.
 
 ## Corresponding Unit Tests
 - `realtime/unit/connection/connection_failures_test.md` — RTN15a, RTN15b
@@ -587,6 +587,10 @@ This means implementations should:
 - Otherwise wait for state change events with timeout
 - Fail if timeout expires
 
+In **integration and proxy** specs these timeouts are **wall-clock time** (the test waits on a
+real server — see the *Timeout Strategy* section in `docs/integration-testing.md`); in unit specs they may be
+realised with fake/virtual timers.
+
 ## Timer Mocking
 
 Tests verifying timeout behavior should use timer mocking where practical to avoid slow tests.
@@ -1025,8 +1029,6 @@ realtime/
       connection_open_failures_test.md
       ...
   integration/
-    helpers/
-      proxy.md                # Proxy infrastructure spec
     proxy/
       connection_open_failures.md   # RTN14 tests via proxy
       connection_resume.md          # RTN15 tests via proxy

uts/objects/PLAN.md

@@ -56,12 +56,12 @@ All new test files go in `specification/uts/objects/`.
 | `integration/objects_lifecycle_test.md` | RTO23, RTPO15, RTPO17 (create objects, mutate via PathObject, read back, REST provisioning) | ~6 |
 | `integration/objects_sync_test.md` | RTO4, RTO5, RTO17 (attach, sync sequence, re-attach) | ~4 |
 | ~~`integration/objects_batch_test.md`~~ | ~~Batch API not in current spec revision~~ | — |
-| `integration/objects_gc_test.md` | RTO10, RTLM19 (behavioral GC verification with ADVANCE_TIME) | ~2 |
+| `integration/objects_gc_test.md` | RTO10, RTLM19 (observable tombstone semantics: undefined/null reads, re-create with new objectId; the timer-based GC sweep is unit-tier) | ~2 |
 
 ### Proxy Integration Tests
 | File | Spec Points | ~Tests |
 |------|-------------|--------|
-| `integration/proxy/objects_faults.md` | RTO5a2, RTO7, RTO8, RTO17, RTO20e (sync interruption, mutation buffering during re-sync, server-initiated detach, publish failure on FAILED channel, publish during delayed sync) | ~5 |
+| `integration/proxy/objects_faults.md` | RTO5a2, RTO7, RTO8, RTO17, RTO20e (sync interruption, mutation buffering during re-sync, server-initiated detach, in-flight publish failing when channel enters FAILED during sync wait, publish during delayed sync) | ~5 |
 
 **Totals: ~20 files, ~310 tests**
 
@@ -380,6 +380,6 @@ onMessageFromClient: (msg) => {
 | Batch API deferred | Not included in current spec revision (a397e34); may be added in a future spec update |
 | LiveObject/LiveMap/LiveCounter marked internal but still unit-tested | Direct testing of CRDT logic is essential; public API tests can't cover all edge cases |
 | Test IDs use `objects/unit/` prefix | Matches directory structure, not nested under `realtime/` |
-| Behavioral GC testing via ADVANCE_TIME | Verify GC through observable consequences (value becomes null, object recreatable) rather than internal pool state inspection |
+| Behavioral GC testing tiered by clock control | Unit tier verifies the timer-based sweep with fake timers + ADVANCE_TIME; the integration tier verifies observable consequences only (value becomes undefined/null, object recreatable) since integration tests run on wall-clock time |
 | Table-driven tests for input validation | Use FOR loops over scenario arrays (like ably-js forScenarios) to test all invalid/valid type combinations |
 | Bytes data type coverage | Standard test pool includes "avatar" bytes entry; compact/compactJson/value tests verify base64 encoding |

uts/objects/integration/objects_gc_test.md

@@ -0,0 +1,166 @@
+# Objects GC Integration Tests
+
+Spec points: `RTO10`, `RTLM19`, `RTLM5d2h`, `RTLM7`
+
+## Test Type
+Integration test against Ably sandbox
+
+## Protocol Variants
+json, msgpack
+
+Each test in this file runs once per protocol variant. The `PROTOCOL` variable
+is set to `"json"` or `"msgpack"` for the current run. Client options should set
+`useBinaryProtocol: PROTOCOL == "msgpack"`.
+
+## Purpose
+
+Behavioral verification of tombstone semantics end-to-end against the real
+server: removing a map entry tombstones it (RTLM7), tombstoned entries read
+back as undefined/null (RTLM5d2h), and the same key is recreatable — the
+server assigns a fresh objectId to the replacement object, which is safe
+because tombstoned state is retained for the GC grace period (RTO10).
+
+## Scope
+
+The timer-based GC sweep itself (RTO10a–RTO10c, RTLM19a) is verified at the
+**unit tier** (`objects/unit/realtime_object.md`, RTO10 tests), where the
+clock and the sweep interval are controllable. It is intentionally **not**
+exercised here: integration tests run on wall-clock time against a real
+server (see *Integration timeouts are wall-clock* in
+`docs/writing-derived-tests.md`), and the sweep cadence (RTO10a, ~5 minutes)
+combined with the server-provided grace period (RTO10b, default 24 hours per
+RTO10b3) is not observable within test timeouts. Do not use `ADVANCE_TIME`
+in this file.
+
+> Note: assertions of the form `value() == null` denote the spec's
+> undefined/null absent value (RTLM5d2h); a deriving SDK asserts its
+> language's mapping (e.g. `undefined` in JavaScript).
+
+## Sandbox Setup
+
+Tests run against the Ably Sandbox at `https://sandbox.realtime.ably-nonprod.net`.
+
+### App Provisioning
+
+```pseudo
+BEFORE ALL TESTS:
+  response = POST https://sandbox.realtime.ably-nonprod.net/apps
+    WITH body from ably-common/test-resources/test-app-setup.json
+
+  app_config = parse_json(response.body)
+  api_key = app_config.keys[0].key_str
+  app_id = app_config.app_id
+
+AFTER ALL TESTS:
+  DELETE https://sandbox.realtime.ably-nonprod.net/apps/{app_id}
+    WITH Authorization: Basic {api_key}
+```
+
+### Notes
+- Each test uses a unique channel name
+
+---
+
+## RTO10 - Tombstoned object is recreatable with new objectId
+
+**Test ID**: `objects/integration/RTO10/tombstoned-object-gc-recreate-0`
+
+**Spec requirement:** After an object is tombstoned (removed from its parent
+map), a new object can be created at the same map key. The new object gets a
+different server-assigned objectId, confirming the old object is gone while
+its tombstone is retained for the grace period (RTO10).
+
+### Setup
+```pseudo
+channel_name = "objects-gc-object-" + random_id()
+
+client = Realtime(options: { key: api_key, endpoint: "nonprod:sandbox", autoConnect: false, useBinaryProtocol: PROTOCOL == "msgpack" })
+client.connect()
+AWAIT_STATE client.connection.state == CONNECTED
+  WITH timeout: 15 seconds
+
+channel = client.channels.get(channel_name, { modes: ["OBJECT_SUBSCRIBE", "OBJECT_PUBLISH"] })
+root = AWAIT channel.object.get()
+  WITH timeout: 15 seconds
+```
+
+### Test Steps
+```pseudo
+// Create a counter
+AWAIT root.set("counter", LiveCounter.create(42))
+poll_until(root.get("counter").value() == 42, timeout: 10s)
+
+counter_id = root.get("counter").instance().id
+
+// Remove it (tombstones the entry and the object, RTLM7)
+AWAIT root.remove("counter")
+
+// RTLM5d2h: tombstoned entries read back as undefined/null
+poll_until(root.get("counter").value() == null, timeout: 10s)
+
+// Create a new counter at the same key
+AWAIT root.set("counter", LiveCounter.create(99))
+poll_until(root.get("counter").value() == 99, timeout: 10s)
+```
+
+### Assertions
+```pseudo
+ASSERT root.get("counter").value() == 99
+ASSERT root.get("counter").instance().id != counter_id
+```
+
+### Teardown
+```pseudo
+client.close()
+```
+
+---
+
+## RTLM19 - Tombstoned map entry is re-settable
+
+**Test ID**: `objects/integration/RTLM19/tombstoned-entry-gc-reset-0`
+
+**Spec requirement:** After a map entry is tombstoned (removed, RTLM7), the
+entry can be re-set. The subsequent MAP_SET succeeds because the server
+assigns a newer serial than the removal's, while the tombstoned entry is
+retained for the grace period pending GC (RTLM19).
+
+### Setup
+```pseudo
+channel_name = "objects-gc-entry-" + random_id()
+
+client = Realtime(options: { key: api_key, endpoint: "nonprod:sandbox", autoConnect: false, useBinaryProtocol: PROTOCOL == "msgpack" })
+client.connect()
+AWAIT_STATE client.connection.state == CONNECTED
+  WITH timeout: 15 seconds
+
+channel = client.channels.get(channel_name, { modes: ["OBJECT_SUBSCRIBE", "OBJECT_PUBLISH"] })
+root = AWAIT channel.object.get()
+  WITH timeout: 15 seconds
+```
+
+### Test Steps
+```pseudo
+// Set then remove a key
+AWAIT root.set("ephemeral", "temporary")
+poll_until(root.get("ephemeral").value() == "temporary", timeout: 10s)
+
+AWAIT root.remove("ephemeral")
+
+// RTLM5d2h: tombstoned entries read back as undefined/null
+poll_until(root.get("ephemeral").value() == null, timeout: 10s)
+
+// Set the same key again
+AWAIT root.set("ephemeral", "revived")
+poll_until(root.get("ephemeral").value() == "revived", timeout: 10s)
+```
+
+### Assertions
+```pseudo
+ASSERT root.get("ephemeral").value() == "revived"
+```
+
+### Teardown
+```pseudo
+client.close()
+```

uts/objects/integration/objects_sync_test.md

@@ -172,7 +172,7 @@ client.close()
 
 ---
 
-## RTO4 - Attach without OBJECT_SUBSCRIBE still resolves get() with empty pool
+## RTO4 - Attach without OBJECT_PUBLISH still resolves get() with empty pool
 
 **Test ID**: `objects/integration/RTO4/attach-subscribe-only-0`