Skip to content

Resolve Python measurement refs against the circuit instead of fabricating them - #386

Merged
ciaranra merged 7 commits into
devfrom
fix-batched-measurement-identity
Jul 29, 2026
Merged

Resolve Python measurement refs against the circuit instead of fabricating them#386
ciaranra merged 7 commits into
devfrom
fix-batched-measurement-identity

Conversation

@ciaranra

@ciaranra ciaranra commented Jul 28, 2026

Copy link
Copy Markdown
Member

Part of #382, and fixes #377 for the generated-surface reproduction. Two coupled defects in how measurement records are identified between Python, TickCircuit and DagCircuit.

1. The Python binding fabricated measurement references

extract_tick_meas_refs (dag_circuit_bindings.rs) rebuilt a TickMeasRef from a (tick, gate_idx, qubit) tuple and invented the rest:

record_idx: 0, // Populated by TickCircuit; placeholder for external construction
meas_id: pecos_core::MeasId(0), // Placeholder

PyTickCircuit::detector and PyTickCircuit::observable pass the result straight to TickCircuit::detector / observable, which read record_idx immediately. So every annotation built from Python collapsed all of its references onto record 0. The comment's premise -- that TickCircuit fills the field in later -- does not hold for the only callers there are.

TickCircuit::meas_ref(tick, gate_idx, qubit) now recovers the real reference by reading the MeasId that mz/mz_free stored alongside the qubit; the binding returns a ValueError naming the offending tuple rather than fabricating one.

An intermediate revision instead reconstructed the record index by walking stored ticks and batches. Review caught that this is wrong, and it reproduces: storage order is not allocation order. A later measurement can be appended to an earlier compatible batch, and tick_at permits filling ticks out of sequence. Interleaving mz(q0), mz_free(q1), mz(q2) allocates records 0, 1, 2 but stores [MZ(q0,q2), MeasureFree(q1)], and the reconstruction returned q1 -> 2, q2 -> 1. That is worse in kind than the bug it replaced: the placeholder was obviously wrong, a swapped index is plausibly wrong. Reading the stored id is order-independent and needs no reconstruction.

Known limitation: mz_with_ids lets callers supply external stable ids (Guppy result ids) that are not positional, and for those the record index and the id genuinely differ. TickCircuit does not store the positional ordinal separately, so that case cannot be resolved here. It is tracked in #387 together with the reverse-conversion conflation, which would defeat any fix confined to this side anyway.

2. MeasureFree did not advance the Tick -> DAG record counter

TickHandle::mz_free advances next_meas_record exactly like mz, but the conversion counted MZ only, so later records became unmappable and the annotation remap silently dropped them.

These two must land together. Fixing only the binding produces correct record ordinals that the conversion then cannot resolve -- trading "collapsed onto record 0" for "dropped entirely", which is not an improvement. An earlier revision of this PR narrowed to the binding alone; review caught that the narrowed form was not a safe intermediate state, and this restores the pairing.

Verified end to end

Built the extension into an isolated venv and compared d=3 surface DEMs against Stim:

from_circuit mechanisms matches Stim?
dev 86 (Z) / 80 (X) no
binding fix alone 76 no -- 13 differ, all L0 membership
this branch 76 in all four configs yes, exact match

cx and szz with physical prefixes, Z and X basis.

Testing

Python-level regressions, which is what the previous revisions lacked. Both were verified to fail against a placeholder-reverted build and pass against this one:

=== placeholder build:  2 failed
=== fixed build:        2 passed
  • test_observable_over_two_measurements_uses_both -- an observable over two measurements must yield error(0.375) L0 (2 * 0.25 * 0.75, odd parity of two independent p_meas errors). Collapsed references cancel by XOR parity and give a different probability or no L0 at all.
  • test_measurement_ref_from_another_circuit_is_rejected -- a tuple that identifies no measurement raises ValueError instead of being fabricated.

Rust tests, each mutation-verified to be killed by reverting its change: meas_ref_recovers_what_mz_returned, meas_ref_rejects_refs_that_name_no_measurement, meas_ref_record_ordinal_is_independent_of_meas_id, measure_free_consumes_a_record_ordinal, annotation_records_survive_conversion_across_measure_free.

meas_ref_is_independent_of_batch_and_tick_storage_order covers the interleaving case above and kills the reconstruction approach. It was added only after review found that defect -- the earlier tests all built circuits sequentially and could not see it.

  • cargo test -p pecos-quantum -p pecos-qec green: 774 pecos-quantum lib tests, 0 failed across both.
  • cargo check --workspace --all-targets, cargo clippy -p pecos-quantum -p pecos-rslib --all-targets, cargo fmt --all --check clean.

Deliberately not fixed here -- tracked in #387

  • MeasureLeaked is a measurement in Gate::validate and in DAG→Tick, but not in Tick→DAG or meas_ref.
  • Unresolvable annotation records are still silently dropped in three places. Making the tick_circuit.rs site panic was tried and reverted: panicking inside impl From<&TickCircuit> for DagCircuit is not a safe resolution for an infallible std conversion trait.
  • The identity/ordinal conflation persists in the DAG→Tick direction.
  • Detector annotations reach the DEM only through the sampler path, which this binding fix also corrects but which has no Python regression here.

@ciaranra

Copy link
Copy Markdown
Member Author

Two independent deep reviews of 40b915697 -- Fable and Sol (gpt-5.6-sol), each unaware of the other's findings. Both returned REVISE, and both independently identified the same blocker, each verifying it by running rather than reading. I reproduced it a third time.

The blocker: the conversion still maps by storage order

meas_ref is now allocation-order-correct. The Tick->DAG conversion is not -- it rebuilds meas_record_to_node with a running counter over stored ticks and batches (tick_circuit.rs:3451), ignoring the meas_ids on each split gate. I fixed the lookup and left the mapping.

Reproduced with mz(&[0]), mz_free(&[1]), mz(&[2]) -- allocation records 0,1,2, storage [MZ(q0,q2), MeasureFree(q1)]:

observable over record 2 (qubit 2's MZ)
  -> targets node 5, gate = MeasureFree, qubits = [1]

Wrong qubit, wrong gate type, silent.

Fable added two things I had not established:

  • It is reachable from the plain Python API with no tick_at: t.mz([0]); t.mz_free([1]); t.mz([2]) on one tick handle. Not an exotic construction.
  • This PR makes it worse. Before, the same scenario silently dropped record 2 under MZ-only counting. Fixing the counter converts a drop into a silent wrong answer.

Both reviewers also pinned why my tests missed it: meas_ref_is_independent_of_batch_and_tick_storage_order builds the interleaved circuit but stops before converting, and annotation_records_survive_conversion_across_measure_free converts but uses a circuit where storage order equals allocation order and only asserts the nodes are distinct, never that they are the right ones. The two tests do not compose.

What both reviews confirmed is sound

Worth recording, since it is most of the change:

  • meas_ref's contract holds. position() first-match is safe because find_conflicts plus Gate::validate reject duplicate qubits within a command; meas_ids stays index-parallel to qubits across batch merging because can_batch_with enforces presence parity and length equality; id-less external batches cannot merge with id-carrying ones and fail loud.
  • MeasureLeaked is excluded in the same direction by both meas_ref and the forward conversion -- consistent, not contradictory.
  • All seven tests are real. Fable ran the mutations itself: MZ-only revert fails exactly the conversion test; record-0 fabrication fails three Rust tests; the round-3 storage-order reconstruction fails exactly the storage-order test; and reverting the binding placeholder (with a rebuilt extension) fails both Python tests, with the 0.375 assertion collapsing to logical_observable L0 and no error lines.
  • The mz_with_ids deferral is genuinely unfixable on this side alone, as claimed.

Also found, now tracked in #387

Where this leaves the PR

Five review rounds, five real defects, three introduced by me while fixing the previous one. The pattern is consistent: I fix the site I am looking at and do not check the other half of the same invariant.

Both reviewers propose the same small fix -- key meas_record_to_node on split_gate.meas_ids when present, falling back to the counter for id-less gates, mirroring meas_ref and the reverse conversion. That is a real option. But given the record, I am not proposing another self-directed increment; this may be better folded into #387 and done deliberately.

@ciaranra
ciaranra merged commit 6264f84 into dev Jul 29, 2026
29 checks passed
@ciaranra
ciaranra deleted the fix-batched-measurement-identity branch July 29, 2026 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

from_circuit and generate_dem_from_tick_circuit produce different DEMs for the same circuit

1 participant