compression.md

Compression

const db = new SencilloDB({ file: "./app.json", compression: true });

Persisted files are gzipped. Nothing else about the API changes — data in memory and the documents you get back are unaffected.

What it does to each mode

Mode	Files
Single file	app.json holds gzip data (the extension does not change)
Folder	users.json.gz per collection
Sharded	shard_us.json.gz per bucket; meta.json stays plain
AOF	the log stays plain text so a torn tail can still be recovered

Compressed stores are written and read as streams — the whole file is never held in memory as one string — and the same temp-file-and-rename applies, so a crash cannot leave a half written archive.

The trade-off

Gzip usually shrinks JSON documents by roughly 5–10×, and costs CPU on every read and write. In single file mode that cost is paid on every commit for the whole store, which is why compression is most useful in folder or sharded mode, where only the touched files are rewritten.

Run npm run bench to see the numbers on your data.

Turning it on for an existing store

The flag must match how the files were written; there is no auto-detection. To convert:

const plain = new SencilloDB({ folder: "./data" });
const packed = new SencilloDB({ folder: "./data-gz", compression: true });

await packed.import(await plain.export());

Corrupt archives

A truncated or damaged gzip file raises CorruptDataError on load rather than hanging:

import { CorruptDataError } from "sencillodb";

try {
  await db.transaction((tx) => tx.find({ collection: "users", filter: {} }));
} catch (error) {
  if (error instanceof CorruptDataError) restoreFromBackup();
  else throw error;
}