Skip to content

ENH: seed Monte Carlo per simulation index for worker-invariant inputs - #15

Draft
thc1006 wants to merge 1 commit into
developfrom
enh/per-index-mc-seeding
Draft

ENH: seed Monte Carlo per simulation index for worker-invariant inputs#15
thc1006 wants to merge 1 commit into
developfrom
enh/per-index-mc-seeding

Conversation

@thc1006

@thc1006 thc1006 commented Jun 28, 2026

Copy link
Copy Markdown
Member

This makes Monte Carlo input sampling reproducible regardless of how the run is executed. Today the stochastic models are seeded once per worker (parallel) or once before the loop (serial), so the inputs you get depend on the worker count and on which worker draws which simulation. Two runs with the same seed but different worker counts produce different inputs.

The change seeds per simulation index instead. From the run's root seed I spawn one SeedSequence child per index, and reseed the models from child_seeds[i] before simulation i. SeedSequence.spawn is invariant to the spawn count, so index i always maps to the same child, including in append mode. Each index seed is then split three ways so the environment, rocket, and flight don't share a draw stream.

After this, the generated inputs are identical across serial, parallel(2), and parallel(N) for a fixed seed, and a split append run (3, then +3) matches a single run of 6.

Behavior changes worth calling out

Both shift the numbers a fixed seed produces, so any stored baseline regenerates:

  1. The env/rocket/flight decorrelation. They used to share one seed, so their first draws were correlated.
  2. The serial index field now counts from 0, matching the parallel path, which already did.

Tests

New tests/unit/simulation/test_montecarlo_determinism.py: serial reproducibility, worker-invariance (serial == parallel(2) == parallel(4), bit-identical per index), append reproducibility, and the None-seed path. They stub Flight so only input sampling runs, and drop the one stdlib-random thrust_source so the inputs are byte-reproducible from the numpy seed alone. Marked slow (each sim rebuilds a rocket), matching the existing test_monte_carlo_simulate.

Locally: ruff and pylint both 10/10, and the four tests pass with --runslow.

Known gap

List-valued stochastic attributes are drawn with the stdlib random.choice (a global, unseeded generator), so a model with multi-element lists isn't fully worker-invariant yet. BPC isn't affected (its lists are single-element). A general fix would seed the stdlib random per index too, or move those draws onto numpy. I'll note it upstream when this goes to RocketPy.

Unrelated, but flagging it since I ran the suite: test_monte_carlo_simulate is already failing on develop (processed_results has no "apogee", and the sims themselves don't error), independent of this change. Happy to look at that separately if it's useful.

MonteCarlo seeded the stochastic models once per worker (parallel) or once
before the loop (serial), so the generated inputs depended on how many workers
ran and which worker drew which simulation. Two runs of the same seed with
different worker counts produced different inputs.

Seed per simulation index instead: spawn one SeedSequence child per index from
the run's root seed, and reseed the stochastic models from child_seeds[i]
before simulation i. SeedSequence.spawn is invariant to the spawn count, so
index i always maps to the same child, including in append mode. Each index
seed is split three ways so the environment, rocket and flight do not share a
stream.

Inputs are now identical across serial, parallel(2) and parallel(N) for a fixed
seed, and a split append run reproduces a single run. This changes the numbers a
fixed seed produces (the per-model decorrelation, plus a serial index that now
counts from 0 like the parallel path already did), so stored baselines
regenerate.

Adds tests/unit/simulation/test_montecarlo_determinism.py covering serial
reproducibility, worker-invariance, append reproducibility, and the None-seed
path. Marked slow (each simulation rebuilds a full rocket) per the existing
test_monte_carlo_simulate convention.
@zuorenchen

Copy link
Copy Markdown
Member

Let's pull this from RocketPy once RocketPy-Team#1054 is merged

@thc1006

thc1006 commented Jul 18, 2026

Copy link
Copy Markdown
Member Author

Let's pull this from RocketPy once RocketPy-Team#1054 is merged

for sure
but upstream not yet merged should we wait or implement it ahead of them

@zuorenchen

Copy link
Copy Markdown
Member

I'd prefer to wait for them to reduce conflicts when pulling changes from RocketPy in the future. Also, we'll probably stay with serial monte-carlo for now

@thc1006

thc1006 commented Jul 29, 2026

Copy link
Copy Markdown
Member Author

Marking this draft to match what you asked for on this PR: wait for RocketPy#1054 and pull the implementation back through the usual sync. It is still open, at 8 commits and 9 files, so there is nothing to pull yet.

Leaving it open and unmarked was costing review attention it should not be: this branch is one commit from July 18 and is now the older, narrower version of work that moved upstream. Two things worth recording before it sits, since they are true of this branch and would matter if anyone picked it up later:

Neither reaches BPC today, since it runs Monte Carlo with parallel=False.

One thing that does not depend on this PR and is worth its own issue at some point: StochasticModel.dict_generator and StochasticRocket._randomize_position sample list-valued attributes through the process-global random.choice, which no seed controls. That is on develop now, not something this branch introduces, and it means a model with a multi-element list is not reproducible from a seed even in serial. I have not measured whether BPC's balloon model has one; I will check and open an issue if it does.

@thc1006
thc1006 marked this pull request as draft July 29, 2026 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants