stepSize: all tooling to use tx.Debug().StepSize() instead of DefaultStepSize constant#20280
Conversation
…ultStepSize` constant (#20280)
There was a problem hiding this comment.
Pull request overview
This PR updates integration tooling to use the configured ErigonDB step size (via tx.Debug().StepSize() / state.ResolveErigonDBSettings) instead of hardcoding config3.DefaultStepSize, and enhances integration print_stages output to show more granular domain history pruning/index progress.
Changes:
- Update integration commands to derive step size from DB settings / transaction debug info rather than
DefaultStepSize. - Add helper logic for opening/scanning history with resolved ErigonDB settings.
- Extend
print_stagesto print per-domain/per-table index step counts for pruning/progress visibility.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| db/config3/config3.go | Clarifies comments around legacy/default step-size constants and encourages reading actual settings from erigondb.toml. |
| cmd/integration/commands/state_history.go | Refactors history tooling to use ResolveErigonDBSettings and step-size derived values (instead of config constants). |
| cmd/integration/commands/reset_state.go | Enhances printStages output to include additional index-step metrics per inverted index table. |
| cmd/integration/commands/commitment.go | Replaces DefaultStepSize usage with tx.Debug().StepSize() for commitment reads/benchmarks. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if err != nil { | ||
| return nil, nil, fmt.Errorf("init history: %w", err) | ||
| } | ||
| history.Scan(scanToStep * settings.StepSize) |
There was a problem hiding this comment.
History.Scan returns an error, but the result is currently ignored. This won’t compile (unused return value) and also drops scan failures at runtime. Capture and handle the error (return it from openHistory with context).
| history.Scan(scanToStep * settings.StepSize) | |
| if err := history.Scan(scanToStep * settings.StepSize); err != nil { | |
| return nil, nil, fmt.Errorf("scan history: %w", err) | |
| } |
| func openHistory(dirs datadir.Dirs, domainName string, scanToStep uint64, logger log.Logger) (*state.History, *state.ErigonDBSettings, error) { | ||
| settings, err := state.ResolveErigonDBSettings(dirs, logger, false) | ||
| if err != nil { | ||
| return nil, nil, fmt.Errorf("resolve erigondb settings: %w", err) | ||
| } | ||
| domainKV, err := kv.String2Domain(domainName) | ||
| if err != nil { | ||
| return nil, nil, fmt.Errorf("resolve domain: %w", err) | ||
| } | ||
| history, err := state.NewHistory( | ||
| statecfg.Schema.GetDomainCfg(domainKV).Hist, | ||
| settings.StepSize, | ||
| settings.StepsInFrozenFile, | ||
| dirs, |
There was a problem hiding this comment.
ResolveErigonDBSettings can read values from erigondb.toml without validation; if StepSize or StepsInFrozenFile is 0, this code will later divide by zero (e.g., fromTxNum/stepSize) or loop forever. Add a sanity check in openHistory (e.g., require both > 0) and return a clear error if invalid.
| err = roTx.HistoryDump( | ||
| int(fromStep)*config3.DefaultStepSize, | ||
| int(toStep)*config3.DefaultStepSize, | ||
| int(fromStep)*int(stepSize), | ||
| int(toStep)*int(stepSize), | ||
| keyToDump, |
There was a problem hiding this comment.
fromStep/toStep are uint64 (default --to is 1e18). Converting to int and multiplying by stepSize can overflow int (and also overflow the uint64 step->txNum conversion earlier), producing negative/incorrect ranges passed to HistoryDump. Consider clamping toStep based on math.MaxInt/stepSize (or using a sentinel like --to=0 meaning “max”) and validating before conversion.
…x, xsync fix, warmuper fix, logs debug, unused code (#20337) Cherry-picks from release/3.4 to main: - integrity: blk-range chk to pre-build "changed keys" index before calc state root (#20302) - up gql and grpc (#20304) - stepSize: all tooling to use `tx.Debug().StepSize()` instead of `DefaultStepSize` constant (#20280) - db/state: fix CursorHeap tie-break to prefer RAM over DB over FILE (#20318) - xsync deprecated use fix (#20331) - warmuper.WaitAndClose: cancel work before wait (#20332) - logs: move some Info logs to Debug level (to simplify logs for Users) (#20329) - db: unused code remove (#20334) --------- Co-authored-by: moskud <sudeepdino008@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
tx.Debug().StepSize()instead ofDefaultStepSizeconstantintegration print_stagesto show each domain history pruning progress (separately for each table)