fix(plugin): correct replay callbacks and operation span timing

[zhongkechen]

Summary

• stop the plugin executor from invoking on_operation_start or on_operation_end for terminal operations encountered during replay
• continue invoking on_operation_start for replayed operations that are still non-terminal
• remove terminal-replay filtering from both OTel plugins so the behavior is guaranteed by the SDK plugin interface
• start invocation-view operation spans with the current time so continuation spans remain within their invocation

Root cause

PluginExecutor.on_operation_replay unconditionally emitted on_operation_start and then also emitted on_operation_end for terminal checkpointed operations. That made every plugin responsible for filtering replay history and caused OTel to emit an already-successful submitter again.

The executor now returns before dispatching either callback when the replayed operation has reached SUCCEEDED, FAILED, TIMED_OUT, CANCELLED, or STOPPED. This aligns the Python plugin lifecycle with the TypeScript behavior.

Invocation-view operation spans also reused durable timestamps from prior invocations. They now use the current local time for both start and end so continuation spans do not overlap their invocation parent.

Conformance failures:

• https://github.com/aws/aws-durable-execution-conformance-tests/actions/runs/30304154081/job/90104146029
• https://github.com/aws/aws-durable-execution-conformance-tests/actions/runs/30310125440/job/90123537458

Testing

• hatch run test:all (3062 passed, 2 skipped, 5 subtests passed)
• plugin executor replay tests (2 passed, 5 subtests passed)
• replay-flow state tests (7 passed)
• hatch run dev-otel:test (92 passed)
• hatch run types:check
• hatch fmt --check for core and OTel packages

[github-actions]

Codex AI review

No actionable findings. Residual risk: this was a static review; repository tests were not executed per instructions.

Reviewed commit 63c2723aba31271884f369bfe9eb3cc8682f1385. Workflow run

[github-actions]

Claude AI review

I reviewed the full diff for PR #591 against the base revision, tracing the changed behavior through plugin.py, state.py, context.py, concurrency/executor.py, and both OTel plugins.

Core change (PluginExecutor.on_operation_replay) — returning early for terminal operations and no longer emitting on_operation_end during replay is correct and preserves exactly-once completion semantics:

• Operations that complete within an invocation still fire on_operation_end via on_operation_update (state.py:847).
• Operations that complete during suspension are carried in UpdatedOperationIds and routed to emit_operation_update_hook → on_operation_end (context.py:556-562), not to the replay path. is_operation_updated_since_last_invocation correctly separates these from replayed history.
• Operations that completed in a prior invocation already emitted their end there; suppressing a duplicate on replay is the intended fix and matches the documented TypeScript behavior.

The non-terminal replay path still fires on_operation_start, and _is_terminal_status covers all five terminal states, matching the parametrized tests.

OTel timing change — switching operation-span start/end to datetime.now(UTC) (invocation_plugin.py:391,419) and dropping the microsecond-bump keeps continuation spans within their invocation parent. Because spans are observability-only (never checkpointed), the non-determinism is not a concern, and the two independent clock reads make zero-duration spans effectively impossible, so removing the +1µs guard is safe. The if not span → if span is None change is equivalent here.

Tests — coverage is adjusted consistently: the deleted test_replayed_operation_start_emits_continuation_span_with_link exercised a path the SDK no longer drives, and the surviving test_retried_operation_start_emits_continuation_span_with_link still covers the existed=True continuation-span-with-link behavior. Docstring updates accurately reflect the new lifecycle.

No actionable findings.

Residual test risk:

• No test asserts the cross-invocation on_operation_end path (span not started in the current invocation, is_replayed=False) produces a linked continuation span with now()-based times and verifies the link — invocation_plugin.py:419 is only indirectly exercised.
• The ExecutionOtelPlugin (execution_plugin.py:394-395,503-504) still retains its own end_time == start_time microsecond-bump, which was not removed unlike InvocationOtelPlugin. This is a pre-existing inconsistency, not introduced by this PR, but worth noting if uniform timing across the two OTel plugins is intended.

Reviewed commit 63c2723aba31271884f369bfe9eb3cc8682f1385. Workflow run