Skip to content

flatkv migration testings test#3462

Open
blindchaser wants to merge 78 commits into
mainfrom
yiren/fkv-mig-int
Open

flatkv migration testings test#3462
blindchaser wants to merge 78 commits into
mainfrom
yiren/fkv-mig-int

Conversation

@blindchaser
Copy link
Copy Markdown
Contributor

Describe your changes and provide context

Testing performed to validate your change

@cursor
Copy link
Copy Markdown

cursor Bot commented May 19, 2026

PR Summary

High Risk
High risk because it rewires CompositeCommitStore initialization/routing and removes ReadMode/EnableLatticeHash, affecting consensus AppHash behavior and state migration paths. Also adds new CI/docker workflows and migration scripts that can change how integration tests and devnets boot and migrate state.

Overview
Updates FlatKV/state-commit configuration to remove sc-read-mode and sc-enable-lattice-hash and introduce sc-keys-to-migrate-per-block, with stricter validation and safer defaults (including early parsing/validation of sc-write-mode).

Refactors sei-db CompositeCommitStore to build all reads/writes/proofs/iterators through the migration Router, make construction/initialization error-returning, handle memiavl/flatkv presence per write-mode, and append the FlatKV lattice hash to CommitInfo based on migration state rather than a config flag.

Expands migration and recovery coverage: adds comprehensive Go tests for MemiavlOnly -> migrate_evm -> evm_migrated, adds a new docker-level FlatKV EVM Migration 0->1 integration job plus a coordinated cutover script, and simplifies existing FlatKV CI scenarios to run with FlatKV enabled from genesis (GIGA_STORAGE=true using test_only_dual_write) while removing the old offline import workflow.

Reviewed by Cursor Bugbot for commit b3b6cc2. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 19, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedMay 20, 2026, 3:41 PM

Comment thread sei-db/state_db/sc/composite/store.go
@codecov
Copy link
Copy Markdown

codecov Bot commented May 19, 2026

Codecov Report

❌ Patch coverage is 62.65487% with 211 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.05%. Comparing base (1dd16d0) to head (b3b6cc2).

Files with missing lines Patch % Lines
sei-db/state_db/sc/composite/store.go 70.60% 50 Missing and 37 partials ⚠️
...b/tools/cmd/seidb/operations/migrate_evm_status.go 0.00% 55 Missing ⚠️
scripts/evm_stress/main.go 0.00% 28 Missing ⚠️
sei-db/state_db/sc/flatkv/store_meta.go 61.90% 14 Missing and 10 partials ⚠️
sei-db/state_db/sc/migration/migration_manager.go 92.50% 3 Missing and 3 partials ⚠️
sei-cosmos/storev2/rootmulti/store.go 20.00% 2 Missing and 2 partials ⚠️
app/seidb.go 0.00% 2 Missing and 1 partial ⚠️
sei-db/config/sc_config.go 50.00% 1 Missing and 1 partial ⚠️
sei-db/tools/cmd/seidb/main.go 0.00% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##             main    #3462    +/-   ##
========================================
  Coverage   59.04%   59.05%            
========================================
  Files        2188     2189     +1     
  Lines      182088   182455   +367     
========================================
+ Hits       107517   107750   +233     
- Misses      64933    65040   +107     
- Partials     9638     9665    +27     
Flag Coverage Δ
sei-chain-pr 64.94% <62.65%> (?)
sei-db 70.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-cosmos/server/config/config.go 94.53% <100.00%> (+0.12%) ⬆️
sei-db/config/write_mode.go 100.00% <100.00%> (ø)
sei-db/state_db/sc/memiavl/store.go 92.59% <100.00%> (-0.06%) ⬇️
sei-db/state_db/sc/migration/router_builder.go 56.72% <100.00%> (+4.40%) ⬆️
sei-db/config/sc_config.go 87.50% <50.00%> (-12.50%) ⬇️
sei-db/tools/cmd/seidb/main.go 0.00% <0.00%> (ø)
app/seidb.go 73.61% <0.00%> (+1.61%) ⬆️
sei-cosmos/storev2/rootmulti/store.go 64.30% <20.00%> (-0.50%) ⬇️
sei-db/state_db/sc/migration/migration_manager.go 94.19% <92.50%> (-1.29%) ⬇️
sei-db/state_db/sc/flatkv/store_meta.go 67.71% <61.90%> (-5.73%) ⬇️
... and 3 more

... and 33 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 75ffaf1428

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

HistoricalProofBurst int `mapstructure:"historical-proof-burst"`

// The number of keys to migrate from memiavl to flatkv per block. Ignored if not in a migration mode.
KeysToMigratePerBlock int `mapstructure:"keys-to-migrate-per-block"`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Wire the migration batch size into app config

Adding this field does not make it configurable at runtime: the generated app.toml template has no matching key and app/seidb.go only reads the existing FlagSC* options, while the new migration script writes keys-to-migrate-per-block expecting it to take effect. As a result, migrate_* runs configured from app.toml keep using the hard-coded default (and server/config.GetConfig leaves the field at zero in its literal), so operators/tests cannot throttle batches and the 0->1 test silently skips the intended small-batch resume coverage. Please add the TOML key/flag and parse it into StateCommitConfig.

Useful? React with 👍 / 👎.

Comment thread app/seidb.go
@blindchaser blindchaser force-pushed the yiren/fkv-mig-int branch 3 times, most recently from 6370d22 to ad6ce4a Compare May 19, 2026 04:30
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ad6ce4a. Configure here.

Comment thread integration_test/contracts/verify_flatkv_evm_migration_0_to_1.sh
@blindchaser blindchaser force-pushed the yiren/fkv-mig-int branch 3 times, most recently from 49fa7a3 to 2a5e537 Compare May 19, 2026 05:15
@blindchaser blindchaser force-pushed the yiren/fkv-mig-int branch 2 times, most recently from 2364ea4 to 152df79 Compare May 19, 2026 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant