-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Problem
When a saved notebook is reopened, the daemon's new_fresh() in notebook_sync_server.rs deletes the persisted automerge document and creates a fresh empty one — treating .ipynb as the sole source of truth. If the .ipynb is missing or stale (e.g. user forgot to save), the automerge document's content is lost.
PR #884 (autosave) mitigates this by keeping .ipynb current, and we've added snapshot-before-delete as a safety net. But the fundamental design treats automerge docs as disposable for file-based notebooks.
Current model
new_fresh()deletes any existingnotebook-docs/{hash}.automergefor file-based notebooks- A fresh empty automerge doc is created
.ipynbis streamed into the empty doc viastreaming_load_cells()- Automerge doc is persisted on a 500ms debounce, but discarded on next reopen
The automerge document is ephemeral — it exists only while the notebook is open (plus cached between sessions, but overwritten on reopen).
Proposed model
Flip the source of truth: the automerge document accumulates all history and is never deleted.
- On reopen, load the existing automerge doc instead of creating a fresh one
- If
.ipynbexists and is newer than the last automerge persist (by mtime), merge its cells into the existing doc - If
.ipynbis missing, use the automerge state as-is — no data loss - Autosave (PR feat(notebook): autosave .ipynb files on daemon #884) becomes a pure export mechanism — writing
.ipynbfor interoperability with git, Jupyter, VS Code, etc.
Key considerations
External edits
The file watcher currently reloads from .ipynb on external change. Under the new model, external edits would need to be merged into the automerge doc rather than replacing it. This preserves edit history while incorporating changes from other tools.
Document size
AutoCommit::save() produces a compacted single-change representation, so document size grows slowly even with many edits. Benchmarking typical notebook sizes would help validate this. Periodic compaction could be added if needed for very long-lived documents.
Migration
No breaking change. Notebooks without an existing automerge doc would continue to bootstrap from .ipynb (the existing load_or_create code path). The change is purely about what happens when both an automerge doc and .ipynb exist.
Interaction with autosave (PR #884)
Autosave ensures .ipynb is almost always up-to-date, making the "which is newer?" check straightforward. The two features are complementary: autosave keeps the export current, automerge-as-source-of-truth prevents data loss.
Files involved
crates/runtimed/src/notebook_sync_server.rs—new_fresh(),streaming_load_cells(), file watchercrates/runtimed/src/daemon.rs—handle_open_notebook(),needs_loaddecision logiccrates/notebook-doc/src/lib.rs—NotebookDoc::load(),NotebookDoc::new()