docs: design — mcpp.toml build environment, platform-conditional config, build.mcpp

Layered manifest-evolution design (L-1 environment provisioning aligned to xlings
.xlings.json subos/workspace/deps/envs; L0 normal/dev/build host-vs-target; L1
Cargo-style [target.'cfg()'] conditional deps+flags evaluated on the resolved
target triple, + lazy fetch + hash-as-identity; L2 generated_files; L3 build.mcpp
native C++ build program with structured-output + declared-I/O disciplines).
Grounded in mcpp internals + a Cargo/Zig/vcpkg/Bazel/xmake survey + the build-
systems literature (a la carte applicative-graph principle, hermeticity/SLSA).

Author: Sunrisepeak

.agents/docs/2026-06-29-manifest-environment-and-platform-design.md

@@ -0,0 +1,251 @@
+# mcpp.toml: Build Environment, Platform-Conditional Config, and `build.mcpp` (Design)
+
+Date: 2026-06-29
+Status: **Design — for discussion.** Synthesizes a multi-tool survey (Cargo, Zig,
+vcpkg, Bazel, xmake, Conan) and the build-systems literature (*Build Systems à la
+Carte*, PubGrub, hermeticity/SLSA) against mcpp's current internals. Scope:
+`src/manifest.cppm`, `src/config.cppm`, `src/xlings.cppm`, `src/toolchain/*`,
+`src/build/{prepare,plan,flags}.cppm`, `src/libs/toml.cppm`. Companion (consumer
+side): mcpp-index `.agents/docs/2026-06-29-mcpp-native-workspace-ci-design.md`.
+
+## 0. Organizing principle
+
+From *Build Systems à la Carte* (Mokhov/Mitchell/Peyton Jones, ICFP 2018): a build
+system's task graph is either **applicative** (dependencies known *before* running
+a task → statically analyzable, lockable, cacheable, parallel-schedulable) or
+**monadic** (dependencies discovered only by *executing* → maximally flexible, not
+analyzable up front). The whole design follows one rule:
+
+> **Keep the dependency graph applicative. Confine imperative logic to bounded
+> leaves.** Anything that decides *which dependencies/inputs exist* is declarative
+> (TOML); per-package flag/source/codegen choices may be imperative — but never
+> gate the top-level graph.
+
+Five layers result, from the host environment up to the produced artifact:
+
+| Layer | Decides | Evaluated on | Today | Industry anchor |
+|---|---|---|---|---|
+| **L-1 Environment** | toolchains + host build-tools + env vars + per-project sandbox | host | only `index_repos` written; xlings `deps`/`workspace`/`envs`/`subos` unused | xlings (already); venv / nix-shell |
+| **L0 Dependency categories** | normal / dev / build | normal·dev = target; build = host | all three parsed; `[build-dependencies]` inert | Cargo, Conan |
+| **L1 Conditional graph** | per-OS/arch deps & features (+ lazy fetch) | resolved **target** triple | none in `mcpp.toml`; recipe-only host-keyed `mcpp.<os>` | Cargo `cfg()`, Zig `.lazy` + hash |
+| **L2 Static codegen** | synthesized headers/sources | — | `generated_files` (exists) | — |
+| **L3 `build.mcpp`** | host probing / dynamic codegen (**leaf only**) | host build, cfg gated on target | none (recipe `install()` is the *third-party* analog) | Zig `build.zig`, Cargo `build.rs` |
+
+---
+
+## L-1 — Build environment: align `mcpp.toml` to xlings' `.xlings.json`
+
+### Current state
+mcpp's only writer of `.xlings.json` is `seed_xlings_json` (`src/xlings.cppm:1070-1088`)
+and the project variant `ensure_project_index_dir` (`src/config.cppm:661-705`); both
+write **only** `index_repos` / `lang` / `mirror`. Toolchains come from `[toolchain]`
+(`manifest.cppm:1072-1079`) → `xim:gcc@16.1.0` (`toolchain/registry.cppm:146-183`) →
+`xlings install` into the `~/.mcpp/registry` sandbox. Host tools (ninja, patchelf)
+are hard-pinned in `xlings.cppm:33-37`. The `workspace.<pkg>` pin you see in
+`.xlings.json` is written by **xlings/xvm**, not mcpp.
+
+Meanwhile xlings *already* models a full per-project environment (`xvm/README.md:84-92`,
+`core/config.cppm`): `deps` (a project package-set, installed by bare `xlings install`),
+`workspace` (target→version, **per-OS keyed**, resolved project > subos > global),
+per-version `envs`, and named/anonymous `subos` (a project sandbox at
+`<proj>/.xlings/subos/...` with its own `bin/lib/usr`). **mcpp surfaces none of it.**
+
+### Design — `[environment]` materializes 1:1 onto xlings keys
+Do **not** invent new vocabulary. A `mcpp.toml` `[environment]` block maps directly
+to the xlings `.xlings.json` schema and is written into mcpp's existing project file
+`<proj>/.mcpp/.xlings.json` (extend the `index_repos`-only writer at
+`config.cppm:699-705` to also emit `deps`/`workspace`/`envs`/`subos`):
+
+```toml
+[environment]
+subos = "dev"                          # → .xlings.json "subos" (named project sandbox)
+tools = ["make@4", "python@3.13.1"]    # → "deps" (host build-tools; bare `xlings install` provisions)
+[environment.workspace]                # → "workspace" (pin tool versions; per-OS values allowed)
+gcc = "16.1.0"
+[environment.env]                      # → per-tool "envs" applied by xvm shims
+OPENBLAS_NUM_THREADS = "1"
+```
+
+- **`[toolchain]` folds in**: today it installs globally; route it into the project
+  `workspace` so two checkouts on one machine pin different compilers via xvm shims
+  (xlings supports this; mcpp just never seeds it). `[toolchain]` stays as the
+  ergonomic shorthand; `[environment.workspace]` is the general form.
+- **Provisioning** = the already-wired project-mode `build_command_prefix`
+  (`xlings.cppm:716-724`) + bare `xlings install` over the emitted `deps`.
+- **This is purely "surface + writer"** — no new resolution machinery; the
+  materialization target is wired but unused.
+
+### Gap closed
+Per-project toolchain pinning, host build-tools (`make`/`python`/`cmake`) as
+declared deps, per-tool env vars, and a reproducible per-project sandbox — all
+expressible, all backed by existing xlings behavior.
+
+---
+
+## L0 — Dependency categories: keep three axes, wire `build`
+
+Keep Cargo/Conan's **normal / dev / build** trichotomy (mcpp already declares all
+three, `manifest.cppm:251-253`; `[build-dependencies]` is parsed but **inert** — no
+consumer in `prepare`/`plan`). They answer two orthogonal questions:
+
+- **normal vs dev** = *when* linked (always / test-only). Both evaluated against the
+  **target**; both link into a runnable artifact (`mcpp test` resolves dev-deps —
+  `manifest.cppm:44-46`).
+- **normal vs build** = *which machine* it runs on (target / **host**). build-deps
+  do not enter the product; they feed the build itself.
+
+**Three distinct "build-tool" homes — do not conflate:**
+1. **Recipe source-build tools** — `xpm.<os>.deps` in an xpkg recipe (`xim:python`,
+   `xim:make`); host, install-time, resolved by xim/`pkginfo.build_dep`. *This is
+   where `compat.xcb`'s Python lives.*
+2. **Project environment tools** — L-1 `[environment].tools` (host tools the project
+   needs available: cmake, protoc).
+3. **`build.mcpp` libraries** — `[build-dependencies]` (host libraries the native
+   build program links). **Wire this here** — it gains a real consumer at L3.
+
+---
+
+## L1 — Conditional dependency graph (Cargo-style `cfg`, target-evaluated)
+
+### Decision: Cargo-style `[target.'cfg(...)']` tables, trimmed token set
+A declarative table keeps the graph applicative (L0 principle). vcpkg's inline
+`platform=` has no home for flags; a flat `[<os>]` block can't express arch /
+combinations / negation. Cargo's table shape wins and — unlike Cargo — mcpp lets
+the **same predicate namespace carry both deps and flags** (Cargo can't put flags
+in the manifest at all):
+
+```toml
+# exact-triple escape hatch (already exists, highest precedence)
+[target.x86_64-pc-windows-msvc.dependencies]
+detours = "4.0"
+
+# predicate tables — primary mechanism; deps AND flags share the namespace
+[target.'cfg(windows)'.dependencies.compat]
+openblas = { version = "0.3.33", lazy = true }
+
+[target.'cfg(windows)'.build]
+ldflags = ["-Llib", "-llibopenblas"]
+
+[target.'cfg(all(linux, not(arch = "aarch64")))'.build]
+cxxflags = ["-march=x86-64-v2"]
+```
+
+- **Grammar**: Cargo's `all()/any()/not()` over `key = "value"` plus bare aliases
+  `windows`/`unix`/`linux`/`macos`. **Token set trimmed to mcpp's existing
+  dimensions**: `os`, `arch`, `family`, `env` — these *are* `AbiDim`
+  (`toolchain/abi.cppm:34-38`); `parse_abi_capability` already parses
+  `abi:arch=aarch64`. Defer `pointer_width`/`endian`/`vendor` until asked.
+- **Evaluate against the resolved TARGET triple, not host.** `abi_profile(triple)`
+  (`abi.cppm:67-91`) derives `os`/`arch` from any triple. Timing is already correct:
+  `--target` + `[target.<triple>]` resolve at `prepare.cppm:497-560`, *before* dep
+  resolution at `~731` — merge the matching `cfg` tables into `m->dependencies` /
+  `buildConfig.*` in that window. (The recipe-side `mcpp.<os>` merge keys on **host**
+  `xpkg_platform` — wrong for cross-compiles; the `mcpp.toml` feature must not copy
+  that.)
+- **Parser hooks**: extend the `[target]` loop (`manifest.cppm:1154-1171`, today only
+  `toolchain`+`linkage`) to also read `dependencies`/`dev-dependencies`/
+  `build-dependencies`/`build`. The TOML reader already lexes quoted keys
+  (`toml.cppm:191-203`), so `[target.'cfg(windows)'.dependencies]` parses — but
+  iterate `get_table("target")` entries manually (dotted `get_*` mis-splits quoted
+  segments).
+- **Precedence**: exact triple > cfg; multiple matching cfg tables → flags
+  concatenate, conflicting scalar (e.g. linker) → error (Cargo's rule).
+
+### Lazy + content-hash identity (from Zig)
+- **`lazy = true`** on a dependency → fetched only when a build path (after cfg/feature
+  gating) actually requests it (Zig `b.lazyDependency`). A Linux build then **never
+  downloads** a `cfg(windows)` dependency. Generalizes mcpp's platform-specific compat
+  packages directly.
+- **Hash is identity; url is a mirror** (Zig `build.zig.zon`: *"packages come from a
+  hash, not a url"*). Adopting this **root-causes two items in mcpp's history**: mirror
+  auto-detection (GitHub/GitCode/CN are interchangeable *locations* of one hash) and
+  the stale-index-shard "fetch false-failure" (content identity can't be impersonated
+  by a moved shard path). Pair fetch with a content-addressed store.
+
+---
+
+## L2 — Static codegen
+`generated_files` (`manifest.cppm:1975`) already covers codegen-as-data (synthesized
+headers/sources, e.g. compat.zlib's config header, the openblas Windows anchor). No
+change; it is the preferred mechanism whenever the generated content is static.
+
+---
+
+## L3 — `build.mcpp`: a native imperative build program (do it directly)
+
+### Two distinct mechanisms, two audiences
+- **`install()`** (Lua, in an xpkg recipe) — the **third-party package** build hook
+  (builds a dependency from source; `compat.openblas`/`compat.xcb`). Stays.
+- **`build.mcpp`** (C++, in the project) — the **mcpp-native project** build program.
+  **New, built directly** (not deferred). Both get the same two disciplines below.
+
+### Form — Zig's in-language model, in C++
+A `build.mcpp` (e.g. `build/main.cpp`) is itself a tiny mcpp build compiled with the
+**host** toolchain and run before the main build — Zig's `build.zig` model, but in the
+project's own language (C++), so no second language and it dogfoods mcpp. Its
+dependencies are exactly `[build-dependencies]` (L0 item 3 — now with a consumer).
+
+### Discipline 1 — structured output protocol (not global mutation)
+The program communicates only via stdout directives the engine consumes (Cargo's
+`cargo::` model):
+```
+mcpp:link-lib=openblas
+mcpp:link-search=<abs-dir>
+mcpp:cxxflag=-DHAVE_FOO
+mcpp:cfg=has_avx2
+mcpp:generated=<path>          # a source/header to add to the graph
+mcpp:rerun-if-changed=<path>
+mcpp:rerun-if-env-changed=VAR
+```
+It *requests* graph edges; it never silently mutates build state.
+
+### Discipline 2 — explicit declared inputs/outputs (fixes the `.mcpp_ok` blind spot)
+`rerun-if-changed`/`rerun-if-env-changed` give the opaque program declared inputs, so
+incremental builds stay correct. This is the documented fix for mcpp's `.mcpp_ok`
+gap ("process exited 0 ≠ outputs correct"): replace the bare success marker with a
+**declared-input/declared-output contract** — the program (or recipe) records what it
+read and what it produced; the build re-runs iff a declared input changed and treats
+missing declared outputs as failure.
+
+### Constraints (à la carte + supply-chain)
+- **Leaf only**: `build.mcpp` chooses flags/sources/codegen and emits link
+  requirements — it must **not** gate the top-level dependency graph (that stays in
+  L1 cfg tables, applicative). cfg gating of `build.mcpp` itself evaluates on the
+  **target**, but it compiles/runs on the **host**.
+- **Isolation**: treat its execution as a build *action* with declared inputs/outputs;
+  run sandboxed; prefer platform-level isolation (SLSA) over trusting program code —
+  the consensus is script-level sandboxing alone is insufficient. The existing xim
+  Lua sandbox (no `os.curdir`/`files`/`trymkdir`) is the right instinct; extend the
+  same declared-I/O contract to `install()`.
+
+---
+
+## Phasing (recommended order)
+
+1. **L-1 environment** — highest ROI, self-contained, no upstream dep beyond mcpp; the
+   xlings target model already exists. Surface `[environment]` + extend the project
+   `.xlings.json` writer. Fold `[toolchain]` into `workspace`. Wire `[build-dependencies]`.
+2. **mcpp-index workspace** (companion doc) — the first real consumer; exercises L1's
+   need (Windows-only `compat.openblas`).
+3. **L1 conditional graph** — `[target.'cfg()']` deps+flags, target-evaluated; then
+   `lazy` + content-hash identity.
+4. **L3 `build.mcpp`** — native build program with the two disciplines; backport the
+   declared-I/O contract to recipe `install()`.
+
+## Appendix — cross-tool summary
+- **Declarative-table vs imperative-script**: TOML is static data → declarative tables
+  for the graph (Cargo/vcpkg/Bazel); reserve imperative for a separate file
+  (Cargo `build.rs`, Zig `build.zig`). 
+- **Conditional grammar**: de-facto token set across all six tools = os / arch /
+  family / env; Cargo's `all/any/not` is the composable standard.
+- **Host vs target**: Cargo/Zig/Bazel evaluate manifest conditionals on the **target**;
+  build/tool deps run on the **host**. Conflating them breaks cross-compile (cf. mcpp's
+  aarch64/musl bootstrap hazards — bootstrap tools must be host-static).
+- **Resolution**: the field is converging on PubGrub (complete + explainable); consider
+  it (or MVS / content-addressed identity) over ad-hoc backtracking long-term.
+
+### Sources
+Build Systems à la Carte (ICFP 2018); Cargo reference (specifying-dependencies,
+build-scripts, config); Rust Reference (cfg); Zig build system + build.zig.zon +
+cross-compilation docs; vcpkg manifest reference; Bazel configurable attributes;
+Meson reference tables; Conan 2 requirements; PubGrub; Bazel hermeticity; SLSA.