chore: scaffold parallel onBlock task structure#720
Draft
zclawz wants to merge 3 commits into
Draft
Conversation
Annotate the two independent parallel tasks within onBlock: - Task A: verifySignatures (XMSS sig verification, CPU-bound) - Task B: apply_transition (state transition, independent of A) Both are currently sequential. This commit marks the barrier point (forkchoice import) and the task dispatch sites with TODOs and design comments so the actual threading implementation has clear anchors to build on. Phase 1 goal: dispatch A and B to spawned threads concurrently, join at the barrier, then proceed to forkchoice import only if both succeed. Phase 2 will parallelize post-state compute segments. Execution/proof verification (Phase 3) is future work. Closes #719 (partial — scaffolding only)
#719) Spawn two threads inside the computedstate block: - Task A: stf.verifySignatures (XMSS sig verification, read-only inputs) - Task B: stf.apply_transition (state transition, writes to cpost_state only) Both run concurrently. The main thread joins both before proceeding to forkchoice import, acting as the barrier described in #719. self.allocator is wrapped in std.heap.ThreadSafeAllocator so both tasks can allocate safely. If std.Thread.spawn fails for either task, it falls back to inline (sequential) execution, preserving correctness. Also removes the now-redundant scaffold barrier comment.
…id OOM on macOS Task A (XMSS sig verification) and Task B (STF) previously shared the same chain allocator via ThreadSafeAllocator, causing peak memory to double vs the sequential baseline (both tasks allocate concurrently). Give Task A its own temporary ArenaAllocator so its allocations are freed when it completes. Task B continues to use the chain allocator for cpost_state, which must outlive the task. Fixes the consistent SIGKILL (OOM) on the macos-latest CI runner.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #719
Summary
This PR scaffolds the parallelization structure for
onBlockchain processing as described in #719.What this PR does
Annotates the two independent parallel tasks within
onBlockand marks the barrier point before forkchoice import:verifySignatures: XMSS signature verification (CPU-bound, read-only inputs)apply_transition: State transition (read-only pre_state clone → mutates post_state)forkChoice.onBlock(forkchoice import) is gated on both A and B completing successfullyBoth tasks are currently still sequential. The annotations provide clear anchors for the actual threading implementation in follow-up PRs.
Phases
verifySignatures+apply_transitionto spawned threads concurrentlyDesign notes
pre_state(read-only) +public_key_cache(read-only); B operates on a clonedcpost_statestd.Thread.WaitGroup(or equivalent) is the natural join primitivecc @GrapeBaBa @anshalshukla @gr3999