Skip to content

refactor: eliminate CommSync, output broadcasts, and output widget custom message routing #810

@rgbkrk

Description

@rgbkrk

Phase C of #761: Eliminate CommSync, output broadcast, and output widget custom messages

After Phase B (#809) has clients reading widget state from the doc, this phase removes the parallel broadcast paths. This is where the real simplification happens.

Broadcast variants eliminated

Broadcast variant Replacement Notes
CommSync Automerge sync delivers doc.comms to new clients The entire "Phase 1.5" handshake disappears
Comm (for comm_open, comm_msg(update), comm_close) Doc sync carries the state change comm_msg(custom) stays as an event — it's an irreducible stream
Output Doc sync carries manifest hashes Python already ignores this broadcast (#755). Frontend watches doc.
OutputsCleared Doc sync carries the cleared list Same pattern as Output
DisplayUpdate Daemon updates manifest in doc, sync carries it Scan now includes doc.comms[*].outputs for Output widget displays

Net result: 13 broadcast variants → 8.

Output widget simplification

This is the biggest win. Currently, captured outputs use a protocol-within-a-protocol:

  • Daemon routes outputs to Output widget via comm_msg(custom) with {method: "output", output: ...}
  • Frontend receives, accumulates, calls sendUpdate back to sync state
  • clear_output uses {method: "clear_output", wait: bool}

New approach:

  • Daemon writes captured outputs to doc.comms[widget_id].outputs (same blob manifest pipeline as cell outputs)
  • Frontend reads doc.comms[widget_id].outputs — same rendering path as cell outputs
  • clear_output → daemon clears doc.comms[widget_id].outputs in the doc
  • clear_output(wait=True) → daemon buffers, clears + appends atomically on next output
  • No custom messages for Output widgets at all

CommState reduction

CommState currently does two things:

  1. State trackingHashMap<String, CommEntry> for widget state → replaced by doc.comms
  2. Capture routingcapture_contexts / widget_captures for Output widget output routing → stays (this is routing logic, not state)

Reduce CommState to just the capture routing fields:

pub struct OutputCaptureRouter {
    capture_contexts: RwLock<HashMap<String, Vec<String>>>,
    widget_captures: RwLock<HashMap<String, String>>,
}

update_display_data change

The scan in update_output_by_display_id_with_manifests currently iterates doc.cells[*].outputs. After this change, it must also iterate doc.comms[*].outputs to find display_ids in Output widget captures.

Implementation

crates/runtimed/:

  • Remove CommSync from the new-client handshake in notebook_sync_server.rs
  • Remove NotebookBroadcast::CommSync variant
  • Remove NotebookBroadcast::Comm for comm_open/comm_msg(update)/comm_close (keep for comm_msg(custom))
  • Remove NotebookBroadcast::Output, ::OutputsCleared, ::DisplayUpdate
  • Output widget capture: write to doc.comms[widget_id].outputs instead of broadcasting custom message
  • update_display_data: scan doc.comms[*].outputs in addition to cells
  • Rename CommState to OutputCaptureRouter, remove state tracking
  • In IOPub handler: remove output broadcast, rely on doc sync + changed_tx
  • Apply sync-before-event pattern (like fix(daemon): sync doc to peer before forwarding ExecutionDone #797 for ExecutionDone) for execution_started

crates/notebook-protocol/:

  • Remove CommSync from NotebookBroadcast enum
  • Remove or simplify Comm variant (only custom remains)
  • Remove Output, OutputsCleared, DisplayUpdate variants
  • Keep: ExecutionStarted, ExecutionDone, QueueChanged, KernelError, CommCustom, EnvProgress, EnvSyncState, FileChanged

Frontend:

  • Remove CommSync handling from useDaemonKernel
  • Remove Output broadcast handling (already partially done — outputs come via doc sync)
  • Remove Output widget custom message accumulation (method: "output", method: "clear_output")
  • Output widget reads doc.comms[widget_id].outputs — same manifest resolution as cell outputs

Testing

  • New window opens on notebook with widgets → widgets appear (no CommSync)
  • Output widget captures output → appears in widget (via doc sync, not custom message)
  • clear_output(wait=True) inside Output widget → correct behavior
  • update_display_data for display inside Output widget → updates correctly
  • Broadcast lag recovery still works (doc sync catches up)
  • Python confirm_sync() + get_widgets() works without CommSync

Size

Large — touches daemon IOPub handler, broadcast enum, frontend hooks, Output widget rendering.

Part of #761. Depends on #809.

Metadata

Metadata

Assignees

No one assigned

    Labels

    architectureArchitecture proposals and structural changesenhancementNew feature or requestipywidgetsWidget rendering, comm protocol, Output widgetssyncAutomerge CRDT sync protocol

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions