ci(§8.3): wire N11.2/N11.3 into gate, fix stale fuzz matrix, add nightly+soak workflows#58
Open
MikeRoss27 wants to merge 2 commits into
Open
ci(§8.3): wire N11.2/N11.3 into gate, fix stale fuzz matrix, add nightly+soak workflows#58MikeRoss27 wants to merge 2 commits into
MikeRoss27 wants to merge 2 commits into
Conversation
…ak workflows (§8.3) Closes the CI-tooling half of PLAN-NATIVE-ENGINE.md §8.3. Three concrete bugs found and fixed while wiring this, before any new workflow was written: - ci.yml's `gate` job lists its --test files individually (no auto discovery) and was missing both `model_based` and `io_faults` from the previous commit — they were passing locally but dead in CI. Added. - fuzz.yml's nightly matrix referenced `sst_decode_structured`, removed in N8.5, and only covered 5 of the 24 real fuzz targets. Rewritten to match fuzz/Cargo.toml exactly, one entry per target. - `cargo xtask engine-soak` (N7.3) has been documented "nightly usage" since it was written but was never wired into any workflow. New: - crash_consistency.rs's CYCLES is now overridable via BASEMYAI_CRASH_CYCLES (default 20 unchanged, verified the PR gate is unaffected) instead of a hardcoded const, so a nightly job can run a longer "crash loops prolongés" pass without a second copy of the file. - nightly.yml: crash-consistency at 200 cycles (Linux+Windows) + engine_bench at 100k clear+encrypted, archived as a build artifact. - soak-campaign.yml (weekly): `cargo xtask engine-soak` on Linux/Windows/macOS (macOS deliberately absent from gate/nightly for the already-documented 10x Actions-minutes cost; acceptable weekly). workflow_dispatch allows an on-demand 1M-scale run without paying that cost on the scheduled default. Deliberately not done this pass, documented as open rather than faked: key rotation during soak, near-full-disk simulation, handle counting — none has existing engine/xtask instrumentation to wire up (unlike RSS, already in EngineStats/engine_bench); each needs new engine-side code, not CI config. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A prior Python edit script's text-mode write silently converted this file to CRLF (Windows default line separator), unlike every other file in the repo. No content change — normalizes back to LF only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Complete CI-tooling half of PLAN-NATIVE-ENGINE.md §8.3 (multi-tiered CI architecture for production hardening). Addresses three concrete bugs found while wiring N11.2/N11.3:
model_basedandio_faultstests were passing locally but not executed in CI (missing from explicit --test list).sst_decode_structured(removed N8.5) and only covered 5 of 24 real fuzz targets.cargo xtask engine-soak(N7.3) documented "nightly usage" since inception but never in any workflow.Changes
Fixes
--test model_based --test io_faultsto basemyai-engine test command.New workflows
nightly.yml (3:30 UTC daily):
crash-consistency-long(200 cycles viaBASEMYAI_CRASH_CYCLES) on Linux/Windowsengine_benchat 100k clair+encrypted, archived as artifactsoak-campaign.yml (Sundays 4:00 UTC):
engine-soakacross Linux/Windows/macOS (macOS acceptable weekly cost)workflow_dispatchfor on-demand 1M-scale runs without paying the 3×cost by defaultEnv-var override
BASEMYAI_CRASH_CYCLES(default 20, gate unaffected); nightly/soak can override without test file duplication.Out of scope (documented, not deferred silently)
Key rotation during soak, near-full-disk simulation, handle counting — none has existing engine/xtask instrumentation to wire. Each needs new engine-side code, not CI configuration. Listed in
soak-campaign.ymlcomments as future work.Testing
cargo xtask ci✅ (gate smoke test)BASEMYAI_CRASH_CYCLES=1 cargo test -p basemyai-engine --features test-util --test crash_consistency✅ (env var override)🤖 Generated with Claude Code