Skip to content

refactor: stop using log memory store feature flag and migration fields in production#10755

Open
mraszyk wants to merge 2 commits into
masterfrom
mraszyk/decouple-log-memory-store-feature
Open

refactor: stop using log memory store feature flag and migration fields in production#10755
mraszyk wants to merge 2 commits into
masterfrom
mraszyk/decouple-log-memory-store-feature

Conversation

@mraszyk

@mraszyk mraszyk commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

The one-time migration from the legacy CanisterLog store to the LogMemoryStore has completed on all subnets, so production code no longer needs to consult the feature flag or the per-canister migration status. This decouples the runtime logic from those fields while keeping them serialized for downgrade compatibility (unlike #10714, which was reverted).

  • Drop the log_memory_store_feature flag from Config and all plumbing (canister manager config, SystemState::new*, LogMemoryStore::new(), fetch_canister_logs, scheduler). The LOG_MEMORY_STORE_FEATURE / _ENABLED consts are retained but now unused by production logic.
  • Remove the migration scaffolding: the subnet-level logs_migrated metadata, the scheduler migration code (and filter_canister_log_records), the migration-duration metric, and the orphaned set_migrated / clear_migrated methods.
  • Collapse all production is_migrated() branches to the always-migrated path. next_canister_log_record_idx and log_memory_store_migrated (proto 44/66) are still written and round-tripped, and the legacy canister_log store is retained and kept up to date via dual-write, so checkpoints remain readable by replicas that predate the log memory store.

…ds in production

The one-time migration from the legacy `CanisterLog` store to the
`LogMemoryStore` has completed on all subnets, so production code no
longer needs to consult the feature flag or the per-canister migration
status. This decouples the runtime logic from those fields while keeping
them serialized for downgrade compatibility (unlike #10714, which was
reverted).

- Drop the `log_memory_store_feature` flag from `Config` and all plumbing
  (canister manager config, `SystemState::new*`, `LogMemoryStore::new()`,
  `fetch_canister_logs`, scheduler). The `LOG_MEMORY_STORE_FEATURE` /
  `_ENABLED` consts are retained but now unused by production logic.
- Remove the migration scaffolding: the subnet-level `logs_migrated`
  metadata, the scheduler migration code (and `filter_canister_log_records`),
  the migration-duration metric, and the orphaned `set_migrated` /
  `clear_migrated` methods.
- Collapse all production `is_migrated()` branches to the always-migrated
  path. `next_canister_log_record_idx` and `log_memory_store_migrated`
  (proto 44/66) are still written and round-tripped, and the legacy
  `canister_log` store is retained and kept up to date via dual-write, so
  checkpoints remain readable by replicas that predate the log memory store.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mraszyk mraszyk added the CI_ALL_BAZEL_TARGETS Runs all bazel targets label Jul 13, 2026
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mraszyk mraszyk marked this pull request as ready for review July 13, 2026 15:56
@mraszyk mraszyk requested a review from a team as a code owner July 13, 2026 15:56
@zeropath-ai

zeropath-ai Bot commented Jul 13, 2026

Copy link
Copy Markdown

No security or compliance issues detected. Reviewed everything up to 8dccff3.

Security Overview
Detected Code Changes
Change Type Relevant files
Refactor ► rs/config/src/execution_environment.rs
    Remove log_memory_store_feature field from Config and Default implementation
Enhancement ► rs/embedders/src/wasmtime_embedder/system_api/sandbox_safe_system_state.rs
    Always append delta log to log memory store and adjust checkpoint readability note
► rs/embedders/src/wasmtime_embedder/system_api/sandbox_safe_system_state.rs
    Simplify next index/memory limit calculation to use log_memory_store unconditionally
Refactor ► rs/embedders/src/wasmtime_embedder/tests.rs
    Remove reliance on LOG_MEMORY_STORE_FEATURE in test setup
Refactor ► rs/execution_environment/benches/management_canister/canister_logging.rs
    Remove LOG_MEMORY_STORE_FEATURE usage in bench config
Refactor ► rs/execution_environment/src/canister_logs.rs
    Remove log_memory_store_feature parameter from fetch_canister_logs and related response construction
Refactor ► rs/execution_environment/src/canister_manager.rs
    Remove log_memory_store_feature usage in canister creation flow and related config passing
Refactor ► rs/execution_environment/src/canister_manager/types.rs
    Remove log_memory_store_feature field from CanisterMgrConfig
Refactor ► rs/execution_environment/src/execution_environment.rs
    Remove log_memory_store_feature from call to create_isolation/exec environment setup
Refactor ► rs/execution_environment/src/lib.rs
    Remove log_memory_store_feature from ExecutionServices setup
Refactor ► rs/execution_environment/src/query_handler.rs
    Update fetch_canister_logs path to remove log_memory_store_feature parameter
Refactor ► rs/execution_environment/src/scheduler.rs
    Remove log_memory_store_feature from Scheduler and related metrics/state plumbing
Refactor ► rs/execution_environment/src/scheduler/scheduler_metrics.rs
    Remove round_log_memory_store_migration_duration metric
Refactor ► rs/execution_environment/src/scheduler/test_utilities.rs
    Remove passing log_memory_store_feature in test builder
Refactor ► rs/execution_environment/tests/canister_logging.rs
    Remove LOG_MEMORY_STORE_FEATURE usage in tests
Refactor ► rs/execution_environment/tests/canister_logging.rs
    Remove test references to log memory store feature flag
Refactor ► rs/execution_environment/src/scheduler/test_utilities.rs
    Remove log_memory_store_feature from test builder construction

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes the now-obsolete log migration/runtime toggling logic by making LogMemoryStore the unconditional production source of truth, while still dual-writing and preserving serialized fields to keep checkpoints readable for older replicas.

Changes:

  • Removes the log_memory_store_feature flag from ic_config::execution_environment::Config and deletes all runtime branching/plumbing based on it.
  • Deletes migration scaffolding (logs_migrated subnet metadata, scheduler migration pass, migration metrics, and related helpers), collapsing logic to the “always migrated” path.
  • Updates log fetch, metrics, tests, and benchmarks to use LogMemoryStore APIs unconditionally, while keeping legacy canister_log updated via dual-write for downgrade/checkpoint compatibility.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated no comments.

Show a summary per file
File Description
rs/test_utilities/execution_environment/src/lib.rs Removes test builder helpers for toggling the log memory store feature flag.
rs/state_manager/src/checkpoint.rs Always checks the next_idx soft invariant between log_memory_store and canister_log.
rs/replicated_state/src/metrics.rs Records log memory usage/retention from LogMemoryStore unconditionally.
rs/replicated_state/src/metadata_state/proto.rs Removes initialization of transient logs_migrated metadata.
rs/replicated_state/src/metadata_state.rs Removes the logs_migrated transient field and associated struct plumbing.
rs/replicated_state/src/canister_state/system_state/log_memory_store/tests/log_memory_store.rs Updates tests to the new LogMemoryStore::new() signature and removes flag-dependent cases.
rs/replicated_state/src/canister_state/system_state/log_memory_store/tests/canister_lifecycle.rs Updates lifecycle test setup to use LogMemoryStore::new() without flags.
rs/replicated_state/src/canister_state/system_state/log_memory_store/mod.rs Drops FlagStatus dependency, makes new() flag-free and “always migrated”, removes migrated mutators, adds Default.
rs/replicated_state/src/canister_state/system_state.rs Removes feature-flag plumbing from SystemState construction; always constructs LogMemoryStore directly.
rs/replicated_state/benches/log_memory_store.rs Updates benchmark setup to use LogMemoryStore::new() without feature flags.
rs/execution_environment/tests/canister_logging.rs Removes feature-flag-specific integration tests and updates configs/callers accordingly.
rs/execution_environment/src/scheduler/test_utilities.rs Removes passing log_memory_store_feature into scheduler construction.
rs/execution_environment/src/scheduler/scheduler_metrics.rs Removes migration-duration metric from scheduler metrics.
rs/execution_environment/src/scheduler.rs Removes per-round migration logic and the legacy-gap filtering helper + tests.
rs/execution_environment/src/query_handler.rs Simplifies fetch-canister-logs query handling by removing flag parameter plumbing.
rs/execution_environment/src/lib.rs Removes feature-flag plumbing when wiring execution services.
rs/execution_environment/src/execution_environment.rs Updates the replicated management call path to use the new fetch_canister_logs signature.
rs/execution_environment/src/canister_manager/types.rs Removes log_memory_store_feature from canister manager configuration.
rs/execution_environment/src/canister_manager/tests.rs Updates tests to remove feature-flag configuration and builder usage.
rs/execution_environment/src/canister_manager.rs Applies log memory limit logic unconditionally (no longer gated by migration status) and removes flag from system state creation.
rs/execution_environment/src/canister_logs.rs Makes fetch-canister-logs responses always read from LogMemoryStore and removes legacy filtering path.
rs/execution_environment/benches/management_canister/canister_logging.rs Removes feature-flag config usage in benchmarks.
rs/embedders/src/wasmtime_embedder/tests.rs Updates testing constructors to remove LOG_MEMORY_STORE_FEATURE parameter plumbing.
rs/embedders/src/wasmtime_embedder/system_api/sandbox_safe_system_state.rs Always appends deltas to LogMemoryStore and always derives status fields from it; clarifies dual-write intent for legacy store.
rs/config/src/execution_environment.rs Removes log_memory_store_feature from execution environment config and its default initialization.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants