test(discovery): add the 120-entry install manifest - #58
Merged
Conversation
Turns the inventory in tests/README.md into data the harness can execute: every row becomes an entry with a stable id, the platforms it applies to, the block that describes how it arrives, and the fields the collector is expected to get right about it. TOML rather than YAML so the file keeps its comments - the manifest is reviewed like code, and the reasoning beside an entry is most of its value - and so the loader needs nothing but the standard library. Four entry shapes cover all 120 rows, distinguished by which block an entry carries: install, declare, create or state. The recipe family is derived from that block rather than declared beside it, so the two can never disagree about how an entry is executed. Validation happens at load: an unpinned package, a variant naming a base that does not apply on its platform, or an unknown install method fails immediately rather than twenty minutes into a VM run.
lghupan
approved these changes
Aug 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First of a stacked series building the end-to-end harness specified in
Discovery/tests/README.md. This one is the inventory itself — the part that most needs domain review, which is why it lands alone rather than under several thousand lines of Python.What this is
The 120 rows of the test plan, as data the harness can execute. Each entry carries a stable id, the platforms it applies to, how it arrives on a machine, and what the collector is expected to report about it.
manifest.pyis the only reader of those files. Everything downstream works onEntryobjects, so a format change touches one file.Two decisions worth reviewing
TOML, not YAML. The manifest is reviewed like code and the reasoning beside an entry is most of its value, so the format has to keep comments. TOML also parses with
tomllibfrom the standard library, which is what lets the harness run from the test directory with nothing installed.The recipe family is derived, not declared. A row that said
family = "artifact"while carrying aninstallblock would be executed one way and reported another. The block is the declaration.Numbers that should match the plan
All 42 catalog ids have exactly one row. 51 entries —
declare-mcpandartifact— need no installer at all, which is what makes an early automated run substantial rather than a token slice.Verification
Nine tests cover the counts, the per-OS denominators, the family totals, and the load-time rules. The three CI-facing checks — catalog coverage, id contiguity, canary declaration — arrive in the next PR with the workflow job that runs them.