Skip to content

fix(config): apply --service override before disk-usage validation#6594

Draft
hushy wants to merge 2 commits into
mainfrom
lucas.verdonk/searcher-disk-usage-validation
Draft

fix(config): apply --service override before disk-usage validation#6594
hushy wants to merge 2 commits into
mainfrom
lucas.verdonk/searcher-disk-usage-validation

Conversation

@hushy

@hushy hushy commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

A searcher node started with --service searcher could log a misleading "data dir volume too small" warning at startup, comparing its volume against the indexer split-store and ingest queue disk budgets. A searcher does not use those budgets, so the warning was wrong and misleading.

The cause was validation order: the CLI --service override was applied after load_node_config() had already run validation, so the disk-usage check ran against the default all-in-one service set and counted indexer/ingest budgets.

What changed

  • thread the typed service override from the CLI through node-config loading
  • apply it to the resolved service set before validation
  • keep CLI precedence over both QW_ENABLED_SERVICES and the config file, matching the previous post-load behavior
  • leave config loading unchanged when no CLI override is supplied
  • cover a searcher-only override with a regression test that expects zero disk usage when no split cache is configured

Integration-test follow-up

Applying overrides before validation exposed an invalid setup in the CLI integration-test helper. It passed QuickwitService::supported_services(), which includes the standalone compactor, while its config left enable_standalone_compactors disabled. That only appeared to work previously because the override was applied after validation.

The helper now uses the normal default all-in-one service set, which intentionally excludes the standalone compactor. A code comment documents why supported_services() must not be used there.

How was this PR tested?

  • cargo test -p quickwit-cli --test cli --features=metrics -- --test-threads=1 (18 passed, 1 ignored)
  • cargo +nightly fmt --all -- --check
  • git diff --check

A searcher node started with `--service searcher` could log a misleading
"data dir volume too small" warning at startup, comparing its volume
against the indexer split-store and ingest queue disk budgets. A searcher
does not use those budgets, so the warning was wrong and misleading.

The cause was validation order: the CLI `--service` override was applied
after `load_node_config()` had already run validation, so the disk-usage
check ran against the default (all-services) set and counted indexer/ingest
budgets. This threads the service override through `load_node_config_with_env`
into `build_and_validate`, applying it where `enabled_services` is resolved,
before validation. The override takes precedence over the config file and
env vars, matching the previous post-load behavior.

Co-Authored-By: Claude <noreply@anthropic.com>
@hushy hushy force-pushed the lucas.verdonk/searcher-disk-usage-validation branch from 756a09e to f8fb91a Compare July 12, 2026 13:25
@guilload

Copy link
Copy Markdown
Member

I think a better approach would be to move these disk size checks in the init of each individual services:

  1. measure disk size once on start up in quickwit_serve
  2. each service (compactor, indexer, searcher) does check in its routine and warn if necessary

The warning should also be improved to specify on which device the size estimation is based.

@hushy hushy closed this Jul 15, 2026
@hushy hushy reopened this Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants