reading a `u8` array column returns a view into a reused scan buffer (huge footgun)

[DexterKoelson]

Version: spacetimedb 2.7.0 (bindings-typescript)
Severity: Silent data corruption — reads return wrong values, no error thrown

Summary

When a server module reads a column of type array<u8> (t.array(t.u8())), the deserializer returns a Uint8Array that is a zero-copy view over the row iterator's internal scan buffer, not an owned copy of the data.

That buffer is drawn from a shared pool and is reused by the next table scan. As a result, a u8-array value read from one table becomes silently invalid as soon as any other table scan (iter() / filter()) runs — the Uint8Array then reflects whatever bytes the later scan wrote into the recycled buffer.

The value is only safe to read for as long as no other scan has occurred since it was decoded. Because the reference looks like an ordinary Uint8Array, nothing signals to the caller that it has a limited lifetime.

Minimal reproduction

Given two tables where one has a u8-array column:

// table A has a column:  flags: t.array(t.u8())
// table B is any other table

// CORRECT: read the value while its own scan is still live
for (const row of ctx.db.tableA.iter()) {
  console.log(row.flags); // -> correct bytes, e.g. Uint8Array [ 1 ]
}

// WRONG: materialize rows, then read the u8 array AFTER another scan runs
const rows = [...ctx.db.tableA.iter()]; // scan completes; its buffer returns to the pool
for (const row of rows) {
  const related = [...ctx.db.tableB.iter()]; // new scan reclaims that same buffer and overwrites it
  console.log(row.flags); // -> garbage; now aliases tableB's bytes
}

The first loop prints the correct value; the second prints corrupted bytes for the same rows. No exception is raised. Reordering the read to before the tableB scan makes it correct again, which confirms the value is a live view rather than a copy.

Scalar columns (integers, floats, strings, bools) are unaffected because they are decoded into owned JavaScript values. Only the array<u8> fast path returns a view, so it is uniquely exposed.

Root cause

The array<u8> element type takes a dedicated fast path in the deserializer that returns the reader's bytes directly:

• AlgebraicType.makeDeserializer routes Array of U8 to deserializeUint8Array instead of the generic element-by-element array deserializer.

• deserializeUint8Array → BinaryReader.readUInt8Array → readBytes, which constructs the result as a subarray view over the reader's underlying buffer:

  // BinaryReader.readBytes — returns a VIEW, not a copy
  const array = new Uint8Array(
    this.view.buffer,
    this.view.byteOffset + this.offset,
    length,
  );
  return array;

• The row iterator (tableIterator) obtains its scan buffer from a shared pool via takeBuf() at the start of a scan and calls returnBuf() when the generator completes. takeBuf() on the next scan hands out that same buffer and the scan overwrites its contents.

So the returned Uint8Array remains bound to a buffer that the runtime is free to reuse the moment the originating scan finishes.

Note this also differs by access method: index find() uses a separate dedicated buffer, so it does not recycle the iterator pool buffer, whereas iter() and filter() do. That makes the bug appear or disappear depending on which accessor happens to run between decode and read.

Impact

• Any read of a u8-array column that is retained across a subsequent table scan returns wrong data, silently.
• Common, innocuous-looking patterns trigger it: materialize a table's rows with [...ctx.db.x.iter()], then per row look up related rows with another iter()/filter() before reading the u8 array.
• Derived logic is also affected, e.g. value.includes(...) on the array runs against the corrupted view.
• There is no error or warning; the only symptom is incorrect values.

Expected behavior

Reading a column value should yield data with a lifetime independent of the runtime's internal buffer reuse. A decoded array<u8> should either:

1. be an owned copy (like every other column type), or
2. be clearly documented as a borrowed view valid only until the next table access, ideally with a typed wrapper that makes the borrow explicit rather than returning a bare Uint8Array.

Option 1 (copy on decode) matches the behavior of all other column types and removes the footgun entirely; the extra copy is small relative to the surprise it prevents.

Workaround

Snapshot the value into an owned array immediately after reading it, before any other table scan:

const flags = Array.from(row.flags); // owns its bytes; safe across later scans
// or: const flags = row.flags.slice();