Skip to content

feat(notebook-doc): add comms/ map to Automerge schema (v3) with dual-write from daemon #808

@rgbkrk

Description

@rgbkrk

Phase A of #761: Schema + dual-write

Add a comms/ top-level map to the notebook Automerge document so widget state is replicated via the same CRDT sync that handles cells and metadata. This is a pure addition — no behavior changes, existing CommState + CommSync continue to work as fallback.

Schema change

ROOT/
  schema_version: u64              ← bump to 3
  ...existing cells/, metadata/...
  comms/                           ← Map keyed by comm_id (NEW)
    {comm_id}/
      target_name: Str
      model_module: Str            ← e.g. "@jupyter-widgets/controls"
      model_name: Str              ← e.g. "IntSliderModel"
      state: Str                   ← JSON-encoded widget state
      outputs/                     ← List<Str> (OutputModel only: manifest hashes)
      seq: u64                     ← Insertion order counter

Precedent

This follows the same pattern as the native metadata migration (#791):

  1. Add new structured data to the doc
  2. Dual-write (new + old) during transition
  3. Remove old path in a later phase

Implementation

crates/notebook-doc/:

  • Add comms map creation in NotebookDoc::new()
  • Add migrate_v2_to_v3() — adds empty comms map, bumps schema_version to 3
  • Add methods: put_comm(comm_id, target_name, model_module, model_name, state_json, seq), update_comm_state(comm_id, state_json), remove_comm(comm_id), get_comms() -> Vec<CommSnapshot>, clear_comms()
  • For OutputModel: append_comm_output(comm_id, manifest_hash), clear_comm_outputs(comm_id)
  • Tests for all operations

crates/runtimed/ (daemon):

  • In the IOPub handler (kernel_manager.rs), after comm_state.on_comm_open(), also write to doc.comms
  • After comm_state.on_comm_update(), also update doc.comms[comm_id].state
  • After comm_state.on_comm_close(), also remove doc.comms[comm_id]
  • On kernel shutdown (comm_state.clear()), also call doc.clear_comms()
  • Continue sending CommSync broadcast (old path still active)

crates/notebook-protocol/:

  • No changes needed — CommSnapshot struct already exists and can be reused

What this does NOT change

  • Frontend still reads widgets from CommSync broadcast → WidgetStore
  • CommState still exists and is kept in sync
  • Binary buffers still inline (blob store migration is Phase D)
  • No new frame types or protocol changes

Testing

  • Unit tests: add/update/remove comms in NotebookDoc, verify with get_comms()
  • Integration test: daemon writes comm to doc, second client syncs and sees it
  • Migration test: v2 doc loads, migration runs, comms map exists
  • Verify existing widget tests still pass (CommSync path unchanged)

Size

Medium — mostly notebook-doc schema work + daemon dual-write wiring.

Part of #761.

Metadata

Metadata

Assignees

No one assigned

    Labels

    architectureArchitecture proposals and structural changesenhancementNew feature or requestipywidgetsWidget rendering, comm protocol, Output widgetssyncAutomerge CRDT sync protocol

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions