From 112aef1791c633cc4bb4b0cfc826c14c44e527fc Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sun, 10 May 2026 01:33:11 +0800 Subject: [PATCH] =?UTF-8?q?feat(deps):=20multi-version=20mangling=20?= =?UTF-8?q?=E2=80=94=20Level=201=20fallback=20for=20cross-major=20coexiste?= =?UTF-8?q?nce?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When SemVer merge fails (typically two non-overlapping pins like =0.0.1 ⨯ =0.0.2 across two consumers of the same package), instead of hard-erroring the resolver now stages the secondary version's source under `target/.mangled//...` and rewrites its module declarations to a mangled name (`__v__

__mcpp`). The direct consumer's source is staged + rewritten too so its `import X;` points at the mangled secondary BMI. Both BMIs coexist in the build graph at distinct module names; C++23 module attachment isolates their internal symbols, so no namespace mangling is needed. Touches: - `src/pm/mangle.cppm` (new) — pure helper: name format + a regex-free source rewriter that handles `(export )?module X;`, `(export )?module X:Y;`, `(export )?import X;`, `import X:Y;`. Bare partition imports (`import :Y;`) and the global module fragment (`module ;`) are left intact. 11 new unit tests in `tests/unit/test_mangle.cpp`. - `src/cli.cppm` — resolver conflict branch: when `try_merge_semver` fails, run mangling instead of returning. WorkItem gains a `consumerDepIndex` field so the fallback can locate the consumer manifest for staging. Resolved record uses the mangled module name as `[package].name` so the modgraph validator's "module must be prefixed by package name" check passes. - `src/build/plan.cppm` + `ninja_backend.cppm` — when two compile units share a basename across packages (e.g. cmdline 0.0.1 and staged-mangled 0.0.2 both have `parse.cppm`), the colliding object paths get a per-package subdirectory (`obj//`). `.ddi` placement follows. Single-package builds keep the original `obj/` layout so cache hashes don't churn. - `tests/e2e/33_multi_version_mangling.sh` (new) — Y-shape graph: libA pinned to cmdline 0.0.1, libB pinned to cmdline 0.0.2; the build succeeds with both BMIs (`mcpplibs.cmdline.gcm` + `mcpplibs.cmdline__v0_0_2__mcpp.gcm`) on disk and the runtime output matches. - `tests/e2e/32_semver_merge.sh` — drop the previously-irreconcilable case; that's now Level 1's territory and is covered by test 33. MVP scope (clear errors out of bounds): - Conflicting consumer must be a dep, not the main package. - Secondary version must be a leaf (no own transitive deps). Both surfaces are documented in the error message so users know which limit they hit and have a workaround (pin one version). CHANGELOG 0.0.3 entry: covers all three resolution levels (transitive walker / SemVer merge / mangling) and the obj-path-namespacing side-fix to the build backend. --- CHANGELOG.md | 30 +++- src/build/ninja_backend.cppm | 8 +- src/build/plan.cppm | 23 ++- src/cli.cppm | 209 ++++++++++++++++++++++--- src/pm/mangle.cppm | 204 ++++++++++++++++++++++++ tests/e2e/32_semver_merge.sh | 57 +------ tests/e2e/33_multi_version_mangling.sh | 122 +++++++++++++++ tests/unit/test_mangle.cpp | 127 +++++++++++++++ 8 files changed, 699 insertions(+), 81 deletions(-) create mode 100644 src/pm/mangle.cppm create mode 100755 tests/e2e/33_multi_version_mangling.sh create mode 100644 tests/unit/test_mangle.cpp diff --git a/CHANGELOG.md b/CHANGELOG.md index e878d1f..ff62523 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,8 +5,8 @@ ## [Unreleased] — 0.0.3 -依赖解析体系的两步演进:0.0.2 release tag 之后合入的 transitive walker -之上,这一版叠加 SemVer 合并;后续 PR 还会补上多版本 mangling 兜底。 +依赖解析体系的三步演进:0.0.2 release tag 之后合入 transitive walker, +这一版补齐 SemVer 合并(Level 2)+ 多版本 mangling 兜底(Level 1)。 ### 新增 @@ -20,10 +20,28 @@ 以不同版本约束声明时,resolver 会把两条原始约束 AND 合并(裸版本号视作 `=X.Y.Z`),向 index 重新查询,选出同时满足两侧的具体版本。若该版本与 此前已 pin 的不一致,旧的 manifest 与 `[build].include_dirs` 会被原地 - 替换为新版本的内容,孩子依赖也按新 manifest 重新入队。完全无重叠 - (典型如 `=0.0.1` 对 `=0.0.2`)仍硬报错并提示后续 PR 会用多版本 - mangling 兜底。新增 e2e `32_semver_merge.sh` 覆盖兼容合并 + 不可调和 - 两条主链路。 + 替换为新版本的内容,孩子依赖也按新 manifest 重新入队。新增 e2e + `32_semver_merge.sh` 覆盖兼容合并 + 不可调和两条主链路。 + +- ✅ **多版本 mangling 兜底(Level 1)** —— SemVer 合并失败时(典型如 + `=0.0.1` ⨯ `=0.0.2` 这种无重叠的 pin),resolver 不再硬报错,而是把次要 + 版本的源码 stage 到 `target/.mangled//...` 下,通过正则改写 + `(export )?module X;` / `(export )?module X:Y;` / `(export )?import X;` + 把模块名替换成 `__v__

__mcpp` 形式,让两个 BMI 在同一构建图 + 里以不同模块名共存(C++23 module attachment 帮我们做 ABI 隔离,无需额外 + namespace mangle)。直接 consumer 的源码也一并 stage + 改写,让它的 + `import` 指向 mangled 副本。MVP 范围:仅处理 dep-as-consumer + 叶子 + secondary 两种情形,主包做 consumer 或 secondary 还有自己的 transitive + deps 时报清晰错误并建议显式 pin。新增 `src/pm/mangle.cppm`(纯改写 + helper + 11 个单元测试)和 e2e `33_multi_version_mangling.sh`。 + +### 改进 + +- 🔧 **构建后端按需为多包做 obj 路径命名空间** —— `plan.cppm` 检测到 + 跨包同名源文件(多版本 mangling 后两个 `parse.cppm` 同时存在的常见情形) + 时,自动把 `obj/.o` 改为 `obj//.o`,`.ddi` + 扫描产物随之放在 object 同目录下。无碰撞时仍是原始 `obj/.o` + 布局,不影响现有缓存命中。 第二个公开版本。新增 C 语言一等公民支持、xpkg 风格依赖命名空间、包管理子系统骨架重构,以及 lib-root 约定。 diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index d046f23..6afddb1 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -357,7 +357,11 @@ std::string emit_ninja_string(const BuildPlan& plan) { if (dyndep) { // ── Phase 1: scan edges (one .ddi per TU). ────────────────────── - // .ddi is placed beside the object: obj/.ddi. + // .ddi is placed beside the object so multi-version mangling can + // namespace by package without producing two `build` rules with + // the same `.ddi` output (plan.cppm switches `cu.object` from + // `obj/.o` to `obj//.o` whenever a basename + // collides across packages — `.ddi` follows that placement). // Skip .c files: they have no `import`s and don't need P1689 scan; // running them through cxx_scan would route them through g++ / // -fmodules which is exactly what C support is here to avoid. @@ -365,7 +369,7 @@ std::string emit_ninja_string(const BuildPlan& plan) { ddi_paths.reserve(plan.compileUnits.size()); for (auto& cu : plan.compileUnits) { if (is_c_source(cu.source)) continue; - auto ddi = (std::filesystem::path("obj") + auto ddi = (cu.object.parent_path() / cu.source.filename()).string() + ".ddi"; ddi_paths.push_back(ddi); append(std::format("build {} : cxx_scan {}\n", diff --git a/src/build/plan.cppm b/src/build/plan.cppm index ed3be84..1d5ceca 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -96,12 +96,33 @@ BuildPlan make_plan(const mcpp::manifest::Manifest& manifest, plan.stdBmiPath = stdBmiPath; plan.stdObjectPath = stdObjectPath; + // 1a. Detect basename collisions across packages (multi-version mangling + // stages a second copy of the same dep, so `parse.cppm` and friends + // can show up twice). For colliding files we namespace the object + // path by the unit's owning package so `obj/.o` doesn't get + // two `build` rules. + std::map basenameCount; + for (auto idx : topoOrder) { + basenameCount[object_filename_for(graph.units[idx].path)]++; + } + auto sanitize_pkg = [](const std::string& s) { + std::string out; out.reserve(s.size()); + for (char c : s) out += (c == '.' ? '_' : c); + return out; + }; + // 1. Compile units in topological order for (auto idx : topoOrder) { auto& u = graph.units[idx]; CompileUnit cu; cu.source = u.path; - cu.object = std::filesystem::path("obj") / object_filename_for(u.path); + const auto fname = object_filename_for(u.path); + if (basenameCount[fname] > 1 && !u.packageName.empty()) { + cu.object = std::filesystem::path("obj") + / sanitize_pkg(u.packageName) / fname; + } else { + cu.object = std::filesystem::path("obj") / fname; + } if (u.provides) { cu.providesModule = u.provides->logicalName; } diff --git a/src/cli.cppm b/src/cli.cppm index c045e79..2a6fb35 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -32,6 +32,7 @@ import mcpp.config; import mcpp.fetcher; import mcpp.pm.resolver; // PR-R4: extracted from cli.cppm import mcpp.pm.commands; // PR-R5: cmd_add / cmd_remove / cmd_update live here now +import mcpp.pm.mangle; // Level 1 multi-version fallback (cross-major coexistence) import mcpp.ui; import mcpp.bmi_cache; import mcpp.dyndep; @@ -1062,11 +1063,15 @@ prepare_build(bool print_fingerprint, }; std::map resolved; + // Sentinel for "the consumer is the main package" (no dep_manifests entry). + constexpr std::size_t kMainConsumer = static_cast(-1); + struct WorkItem { std::string name; // dep map key as written mcpp::manifest::DependencySpec spec; // copy (we may mutate version) std::string requestedBy; // who asked for it std::string originalConstraint; // spec.version BEFORE pinning (for SemVer merge) + std::size_t consumerDepIndex; // dep_manifests slot of who pushed this child; kMainConsumer for main }; std::deque worklist; @@ -1212,15 +1217,76 @@ prepare_build(bool print_fingerprint, } }; + // Stage a dep's source files into a fresh directory, rewriting their + // module / import declarations against `rename`. Used by the multi- + // version mangling fallback (Level 1) so two cross-major copies of + // the same package can coexist with distinct module names. + // + // Headers (referenced via `[build].include_dirs`) are NOT staged — + // those keep pointing at the original install dir via absolutized + // include paths. + auto stage_with_rewrite = [](const std::filesystem::path& srcRoot, + const std::filesystem::path& dstRoot, + const mcpp::manifest::Manifest& depManifest, + const std::map& rename) + -> std::expected + { + std::error_code ec; + std::filesystem::create_directories(dstRoot, ec); + if (ec) return std::unexpected(std::format( + "stage: cannot create '{}': {}", dstRoot.string(), ec.message())); + + // Resolve the source globs against the original root, falling + // back to the convention default if the manifest didn't set any. + std::vector globs = depManifest.modules.sources; + if (globs.empty()) { + globs = { "src/**/*.cppm", "src/**/*.cpp", + "src/**/*.cc", "src/**/*.c" }; + } + std::set sourceFiles; + for (auto const& g : globs) { + for (auto& p : mcpp::modgraph::expand_glob(srcRoot, g)) { + sourceFiles.insert(p); + } + } + if (sourceFiles.empty()) { + return std::unexpected(std::format( + "stage: no source files found under '{}' (globs={})", + srcRoot.string(), globs.size())); + } + + for (auto const& f : sourceFiles) { + auto rel = std::filesystem::relative(f, srcRoot, ec); + if (ec) return std::unexpected(std::format( + "stage: cannot relativize '{}': {}", f.string(), ec.message())); + auto dst = dstRoot / rel; + std::filesystem::create_directories(dst.parent_path(), ec); + + std::ifstream is(f); + if (!is) return std::unexpected(std::format( + "stage: cannot read '{}'", f.string())); + std::stringstream buf; buf << is.rdbuf(); + std::string content = buf.str(); + + std::string out = mcpp::pm::rewrite_module_decls(content, rename); + std::ofstream os(dst); + if (!os) return std::unexpected(std::format( + "stage: cannot write '{}'", dst.string())); + os << out; + } + return {}; + }; + // Seed the worklist from the main manifest. Dev-deps only when the // caller wants them; they're never propagated transitively. const std::string mainPkgLabel = m->package.name; for (auto& [n, s] : m->dependencies) { - worklist.push_back({n, s, mainPkgLabel, s.version}); + worklist.push_back({n, s, mainPkgLabel, s.version, kMainConsumer}); } if (includeDevDeps) { for (auto& [n, s] : m->devDependencies) { - worklist.push_back({n, s, mainPkgLabel + " (dev-dep)", s.version}); + worklist.push_back({n, s, mainPkgLabel + " (dev-dep)", + s.version, kMainConsumer}); } } @@ -1274,21 +1340,124 @@ prepare_build(bool print_fingerprint, item.originalConstraint, fetcher); if (!merged) { - return std::unexpected(std::format( - "dependency '{}{}{}' has irreconcilable versions in " - "the transitive graph:\n" - " '{}' (constraint '{}') requested by '{}'\n" - " '{}' (constraint '{}') requested by '{}'\n" - "SemVer merge: {}\n" - "C++ modules require a single global version of each " - "package; pick a version compatible with both " - "consumers, or ask one upstream to widen its dep " - "range. (cross-major fallback via multi-version " - "mangling is planned in a follow-up PR)", - key.ns, key.ns.empty() ? "" : ".", key.shortName, - it->second.version, it->second.constraint, it->second.requestedBy, - spec.version, item.originalConstraint, item.requestedBy, - merged.error())); + // Level 1 fallback: multi-version mangling. Two + // versions can't be reconciled by SemVer, but they + // can coexist in the same build if we mangle the + // secondary copy's module name and rewrite the one + // consumer that asked for it. The primary keeps its + // authored module name so consumers that don't care + // about the secondary see no churn. + // + // MVP scope (these limits surface as clear errors): + // * The conflicting consumer must be a dep, not + // the main package — main-package mangling + // would mean rewriting user-authored sources, + // which is too surprising for a fallback path. + // * The secondary version must be a leaf (no own + // transitive deps) — recursive mangling is + // deferred to a follow-up. + if (item.consumerDepIndex == kMainConsumer) { + return std::unexpected(std::format( + "dependency '{}{}{}' has irreconcilable versions:\n" + " '{}' (constraint '{}') requested by '{}'\n" + " '{}' (constraint '{}') requested by '{}'\n" + "SemVer merge: {}\n" + "Multi-version mangling can't help here — the conflict " + "involves the main package directly. Pin one version " + "explicitly in your mcpp.toml.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.version, it->second.constraint, it->second.requestedBy, + spec.version, item.originalConstraint, item.requestedBy, + merged.error())); + } + + auto loaded = loadVersionDep(name, spec.version); + if (!loaded) return std::unexpected(loaded.error()); + auto& [secondaryRoot, secondaryManifest] = *loaded; + + if (!secondaryManifest.dependencies.empty()) { + return std::unexpected(std::format( + "dependency '{}{}{}' has irreconcilable versions:\n" + " '{}' requested by '{}'\n" + " '{}' requested by '{}'\n" + "Multi-version mangling fallback only handles leaf " + "secondaries in 0.0.3 — but the secondary v{} declares " + "its own dependencies, which would need recursive " + "mangling. Pin one version explicitly, or wait for " + "the recursive-mangling extension.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.version, it->second.requestedBy, + spec.version, item.requestedBy, + spec.version)); + } + + // Module names in the source files use the dep's full + // [package].name (e.g. "mcpplibs.cmdline"), not the + // namespaced-subtable shortName. Use that for the + // rename key so the rewriter actually matches what the + // .cppm sources declare. + const std::string moduleName = secondaryManifest.package.name; + std::string mangled = + mcpp::pm::mangle_name(moduleName, spec.version); + + // Stage layout: + // /target/.mangled//__/ ← rewritten secondary source + // /target/.mangled//__self__/ ← rewritten consumer source + auto& consumerManifest = *dep_manifests[item.consumerDepIndex]; + auto consumerRoot = packages[item.consumerDepIndex + 1].root; + auto stageBase = *root / "target" / ".mangled" + / consumerManifest.package.name; + auto secStage = stageBase + / std::format("{}__{}", moduleName, spec.version); + auto consumerStage = stageBase / "__self__"; + + std::map rename{ {moduleName, mangled} }; + if (auto r = stage_with_rewrite(secondaryRoot, secStage, + secondaryManifest, rename); !r) + return std::unexpected(r.error()); + if (auto r = stage_with_rewrite(consumerRoot, consumerStage, + consumerManifest, rename); !r) + return std::unexpected(r.error()); + + // Re-anchor the consumer's PackageRoot at its staged copy + // so the modgraph scanner picks up the rewritten imports. + packages[item.consumerDepIndex + 1].root = consumerStage; + + // Record the staged secondary as a brand-new dep entry + // under its mangled name, so future encounters of this + // exact (ns, mangled) pair dedup cleanly. The original + // primary entry (it->second) is untouched. + auto stagedManifest = secondaryManifest; + // Update [package].name to the mangled module name so + // the modgraph validator (which checks "exported module + // must be prefixed by package name") accepts the + // rewritten sources. + stagedManifest.package.name = mangled; + // Absolutize secondary's include_dirs against its original + // install root so the staged copy still finds headers. + for (auto& inc : stagedManifest.buildConfig.includeDirs) { + if (inc.is_relative()) inc = secondaryRoot / inc; + } + + dep_manifests.push_back( + std::make_unique(std::move(stagedManifest))); + packages.push_back({secStage, *dep_manifests.back()}); + auto added = propagateIncludeDirs(secStage, *dep_manifests.back()); + + ResolvedKey mangledKey{key.ns, mangled}; + resolved[mangledKey] = ResolvedRecord{ + .version = spec.version, + .constraint = item.originalConstraint, + .requestedBy = item.requestedBy, + .source = "version", + .depIndex = dep_manifests.size() - 1, + .includeDirsAdded = std::move(added), + }; + + mcpp::ui::info("Mangled", + std::format("{} v{} ↔ v{} → {} (cross-major fallback)", + moduleName, it->second.version, spec.version, mangled)); + continue; } // Combine the constraint strings so future merges AND with @@ -1365,7 +1534,8 @@ prepare_build(bool print_fingerprint, for (auto& [child_name, child_spec] : dep_manifests[it->second.depIndex]->dependencies) { worklist.push_back({child_name, child_spec, newLabel, - child_spec.version}); + child_spec.version, + it->second.depIndex}); } continue; } @@ -1514,9 +1684,10 @@ prepare_build(bool print_fingerprint, key.ns.empty() ? "" : ".", key.shortName, sourceKind == "version" ? spec.version : sourceKind); + const std::size_t selfIdx = dep_manifests.size() - 1; for (auto& [child_name, child_spec] : dep_manifests.back()->dependencies) { worklist.push_back({child_name, child_spec, thisDepLabel, - child_spec.version}); + child_spec.version, selfIdx}); } } diff --git a/src/pm/mangle.cppm b/src/pm/mangle.cppm new file mode 100644 index 0000000..4aac290 --- /dev/null +++ b/src/pm/mangle.cppm @@ -0,0 +1,204 @@ +// mcpp.pm.mangle — multi-version coexistence (Level 1 of the dep +// resolver's three-step strategy). +// +// When the SemVer merger (`try_merge_semver`) fails because two +// transitive consumers of the same package can't share a version, +// the resolver picks one as "primary" (keeps its module name as +// authored) and rewrites the secondary copy under a mangled name so +// both BMIs can coexist in the build graph without ODR clashes. +// +// This module owns the **pure** half of that work: +// * `mangle_name` decides the mangled name format +// * `rewrite_module_decls` does a regex pass over a single .cppm +// file's text, rewriting module / import declarations whose name +// appears in a caller-supplied rename table. +// +// The orchestration half (deciding which package gets mangled, +// staging files into a per-build directory, splicing the staged +// PackageRoot into the resolver output) lives in `cli.cppm`. + +export module mcpp.pm.mangle; + +import std; + +export namespace mcpp::pm { + +// Mangled module name format: `__v____mcpp`. +// The double underscores keep the suffix outside the user namespace +// (C++ ABI reserves `__` for the implementation, so users can't +// collide), and the trailing `__mcpp` makes link-error backtraces +// obviously mcpp-generated rather than mistaken for hand-rolled. +// +// Dots in `version` become underscores so the result is a valid +// C++ module identifier (modules allow `.` but using it here would +// confuse partition-style readings later). +std::string mangle_name(std::string_view base, std::string_view version); + +// Rewrite a single .cppm file's module / import declarations: +// * `(export )?module N;` → `(export )?module rename[N];` +// * `(export )?module N:P;` → `(export )?module rename[N]:P;` +// * `(export )?import N;` → `(export )?import rename[N];` +// * `(export )?import N:P;` → `(export )?import rename[N]:P;` +// +// Names not present in the `rename` table are left intact. Bare +// partition imports (`import :P;`) and the global module fragment +// (`module ;`) are also left intact — they don't name the enclosing +// module so they need no rewriting. +// +// Single-line comments and string literals are not parsed: the +// matcher requires the keyword (`module` / `import`) to be at the +// start of a logical line (whitespace-only prefix). That covers +// every real-world declaration site without needing a full lexer. +std::string rewrite_module_decls( + std::string_view source, + const std::map& rename); + +} // namespace mcpp::pm + +namespace mcpp::pm { + +std::string mangle_name(std::string_view base, std::string_view version) { + std::string vmangled; + vmangled.reserve(version.size()); + for (char c : version) vmangled += (c == '.' ? '_' : c); + return std::format("{}__v{}__mcpp", base, vmangled); +} + +namespace { + +// Module names are dotted identifiers: [A-Za-z_][A-Za-z0-9_.]* +bool is_name_start(char c) { + return std::isalpha(static_cast(c)) || c == '_'; +} +bool is_name_cont(char c) { + return std::isalnum(static_cast(c)) || c == '_' || c == '.'; +} + +// Skip whitespace (spaces, tabs) in `s` from `i`. Returns new index. +std::size_t skip_ws(std::string_view s, std::size_t i) { + while (i < s.size() && (s[i] == ' ' || s[i] == '\t')) ++i; + return i; +} + +// Try to consume the keyword `kw` at position `i`, requiring word boundary. +// Returns the index past `kw`, or std::string::npos on miss. +std::size_t consume_keyword(std::string_view s, std::size_t i, + std::string_view kw) +{ + if (i + kw.size() > s.size()) return std::string::npos; + if (s.substr(i, kw.size()) != kw) return std::string::npos; + std::size_t after = i + kw.size(); + if (after < s.size() && is_name_cont(s[after])) return std::string::npos; + return after; +} + +// Parse a dotted identifier starting at `i`. Returns end index (first +// position past the name) and the name slice. Returns npos if no name. +std::pair +read_name(std::string_view s, std::size_t i) +{ + if (i >= s.size() || !is_name_start(s[i])) return {std::string::npos, {}}; + std::size_t start = i; + ++i; + while (i < s.size() && is_name_cont(s[i])) ++i; + return {i, s.substr(start, i - start)}; +} + +} // namespace + +std::string rewrite_module_decls( + std::string_view source, + const std::map& rename) +{ + if (rename.empty()) return std::string(source); + + std::string out; + out.reserve(source.size()); + + std::size_t i = 0; + while (i < source.size()) { + // We're at the start of a logical line. Capture leading whitespace. + std::size_t lineStart = i; + std::size_t afterWs = skip_ws(source, i); + + // Try to recognize `(export )?(module|import) NAME[:PART][; or ws]`. + // On any deviation, copy the line verbatim. + std::size_t cur = afterWs; + bool hasExport = false; + if (auto p = consume_keyword(source, cur, "export"); p != std::string::npos) { + hasExport = true; + cur = skip_ws(source, p); + } + + std::string_view kw; + if (auto p = consume_keyword(source, cur, "module"); p != std::string::npos) { + kw = "module"; cur = p; + } else if (auto p = consume_keyword(source, cur, "import"); p != std::string::npos) { + kw = "import"; cur = p; + } else { + // Not a module/import line — emit the rest of the physical line. + std::size_t eol = source.find('\n', lineStart); + if (eol == std::string_view::npos) eol = source.size(); + else ++eol; + out.append(source.substr(lineStart, eol - lineStart)); + i = eol; + continue; + } + + std::size_t afterKw = skip_ws(source, cur); + + // Bare partition import / global module fragment: `import :P;` / + // `module ;` — no name to rename, copy verbatim. + if (afterKw < source.size() && (source[afterKw] == ':' || source[afterKw] == ';')) { + std::size_t eol = source.find('\n', lineStart); + if (eol == std::string_view::npos) eol = source.size(); + else ++eol; + out.append(source.substr(lineStart, eol - lineStart)); + i = eol; + continue; + } + + auto [nameEnd, name] = read_name(source, afterKw); + if (nameEnd == std::string::npos) { + // Not a recognized declaration. Verbatim. + std::size_t eol = source.find('\n', lineStart); + if (eol == std::string_view::npos) eol = source.size(); + else ++eol; + out.append(source.substr(lineStart, eol - lineStart)); + i = eol; + continue; + } + + // Look up the name in the rename table. + auto it = rename.find(std::string(name)); + if (it == rename.end()) { + // No rewrite needed for this declaration; copy line verbatim. + std::size_t eol = source.find('\n', lineStart); + if (eol == std::string_view::npos) eol = source.size(); + else ++eol; + out.append(source.substr(lineStart, eol - lineStart)); + i = eol; + continue; + } + + // Emit the rewritten prefix: + // (export )?(module|import) + // Keep whatever follows the name (`:P;` / `;` / extras) verbatim. + out.append(source.substr(lineStart, afterWs - lineStart)); // ws + if (hasExport) out.append("export "); + out.append(kw); out.append(" "); + out.append(it->second); + + // Append the trailing portion (from nameEnd) up to and including + // the newline. + std::size_t eol = source.find('\n', nameEnd); + if (eol == std::string_view::npos) eol = source.size(); + else ++eol; + out.append(source.substr(nameEnd, eol - nameEnd)); + i = eol; + } + + return out; +} + +} // namespace mcpp::pm diff --git a/tests/e2e/32_semver_merge.sh b/tests/e2e/32_semver_merge.sh index 132ca32..06d01d7 100755 --- a/tests/e2e/32_semver_merge.sh +++ b/tests/e2e/32_semver_merge.sh @@ -89,58 +89,9 @@ grep -qE 'Merged.*cmdline.*0\.0\.1|→ v0\.0\.1' build.log || { out="$("$MCPP" run 2>&1 | tail -1)" [[ "$out" == "ok=1" ]] || { echo "unexpected output: $out"; exit 1; } -# ── 2. Irreconcilable case ───────────────────────────────────────────── -# Two non-overlapping exact pins (=0.0.1 vs =0.0.2). The merger fails -# to find a satisfying version and the build hard-errors. The error -# message must mention the package and both constraints so the user -# can pick one. (Cross-major mangling fallback is a separate PR.) - -mkdir -p "$TMP/mylib2" && cd "$TMP/mylib2" -"$MCPP" new mylib2 > /dev/null -cd mylib2 -rm -f src/main.cpp -cat > src/mylib2.cppm <<'EOF' -export module mylib2; -export int mylib2_answer() { return 2; } -EOF -cat > mcpp.toml <<'EOF' -[package] -name = "mylib2" -version = "0.1.0" -[targets.mylib2] -kind = "lib" - -[dependencies.mcpplibs] -cmdline = "=0.0.1" -EOF - -mkdir -p "$TMP/app2" && cd "$TMP/app2" -"$MCPP" new app2 > /dev/null -cd app2 -cat > src/main.cpp <<'EOF' -import std; -import mylib2; -int main() { return mylib2_answer() == 2 ? 0 : 1; } -EOF -cat > mcpp.toml < build-bad.log 2>&1; then - cat build-bad.log - echo "non-overlapping pins should have failed"; exit 1 -fi -grep -q 'irreconcilable versions' build-bad.log \ - && grep -q 'cmdline' build-bad.log \ - || { cat build-bad.log - echo "expected irreconcilable diagnostic missing"; exit 1; } +# Note: the previously-irreconcilable case (=0.0.1 ⨯ =0.0.2 across two +# consumers) used to hard-error here. Level 1 (multi-version mangling) +# now resolves it instead — see tests/e2e/33_multi_version_mangling.sh +# for the cross-major coexistence path. echo "OK" diff --git a/tests/e2e/33_multi_version_mangling.sh b/tests/e2e/33_multi_version_mangling.sh new file mode 100755 index 0000000..4bdf01d --- /dev/null +++ b/tests/e2e/33_multi_version_mangling.sh @@ -0,0 +1,122 @@ +#!/usr/bin/env bash +# 33_multi_version_mangling.sh — Level 1 of dep resolution: when two +# transitive consumers want incompatible (non-overlapping) versions of +# the same package, the secondary copy is rewritten to use a mangled +# module name so both BMIs coexist in the build graph. +# +# Setup: libA pinned to cmdline 0.0.1, libB pinned to cmdline 0.0.2. +# Both live as path-deps of `app`. The SemVer merger has nothing to +# combine (=0.0.1 ⨯ =0.0.2 has no satisfying version), so the resolver +# stages cmdline 0.0.2 + libB under `target/.mangled/` with rewritten +# `module/import` declarations and the build proceeds. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +# Index + xpkgs need to be visible since we exercise version-source deps. +mkdir -p "$MCPP_HOME/registry/data" +if [[ -d "$HOME/.mcpp/registry/data/mcpp-index" ]]; then + ln -sf "$HOME/.mcpp/registry/data/mcpp-index" \ + "$MCPP_HOME/registry/data/mcpp-index" +fi +if [[ -d "$HOME/.mcpp/registry/data/xpkgs" ]]; then + [[ -e "$MCPP_HOME/registry/data/xpkgs" ]] \ + || ln -sf "$HOME/.mcpp/registry/data/xpkgs" \ + "$MCPP_HOME/registry/data/xpkgs" +fi + +# ── libA: pinned to cmdline 0.0.1 ────────────────────────────────────── +mkdir -p "$TMP/libA" && cd "$TMP/libA" +"$MCPP" new libA > /dev/null +cd libA +rm -f src/main.cpp +cat > src/libA.cppm <<'EOF' +export module libA; +import mcpplibs.cmdline; // pulls cmdline 0.0.1 (primary) +import std; +export int libA_v() { return 1; } +EOF +cat > mcpp.toml <<'EOF' +[package] +name = "libA" +version = "0.1.0" +[targets.libA] +kind = "lib" + +[dependencies.mcpplibs] +cmdline = "=0.0.1" +EOF + +# ── libB: pinned to cmdline 0.0.2 (incompatible with libA) ───────────── +mkdir -p "$TMP/libB" && cd "$TMP/libB" +"$MCPP" new libB > /dev/null +cd libB +rm -f src/main.cpp +cat > src/libB.cppm <<'EOF' +export module libB; +import mcpplibs.cmdline; // resolver rewrites this to the mangled secondary +import std; +export int libB_v() { return 2; } +EOF +cat > mcpp.toml <<'EOF' +[package] +name = "libB" +version = "0.1.0" +[targets.libB] +kind = "lib" + +[dependencies.mcpplibs] +cmdline = "=0.0.2" +EOF + +# ── app: pulls both libs ─────────────────────────────────────────────── +mkdir -p "$TMP/app" && cd "$TMP/app" +"$MCPP" new app > /dev/null +cd app +cat > src/main.cpp <<'EOF' +import std; +import libA; +import libB; +int main() { + std::println("a={} b={}", libA_v(), libB_v()); + return libA_v() + libB_v() == 3 ? 0 : 1; +} +EOF +cat > mcpp.toml < build.log 2>&1 || { + cat build.log + echo "mangling build failed"; exit 1; } + +# Sanity: the resolver should announce the mangling step. Either +# `Mangled` (our trace) or the mangled module name landing in the +# ninja file is fine. +grep -q 'Mangled.*cmdline' build.log || { + cat build.log + echo "expected 'Mangled' trace in build log"; exit 1; } + +# Look for the mangled BMI on disk (proof both versions actually built). +find target -name 'mcpplibs.cmdline__v0_0_2__mcpp.gcm' | grep -q . || { + echo "mangled BMI mcpplibs.cmdline__v0_0_2__mcpp.gcm not found" + find target -name '*.gcm' | head -10 + exit 1; } +find target -name 'mcpplibs.cmdline.gcm' | grep -q . || { + echo "primary BMI mcpplibs.cmdline.gcm not found" + find target -name '*.gcm' | head -10 + exit 1; } + +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == "a=1 b=2" ]] || { echo "unexpected output: $out"; exit 1; } + +echo "OK" diff --git a/tests/unit/test_mangle.cpp b/tests/unit/test_mangle.cpp new file mode 100644 index 0000000..f75aed6 --- /dev/null +++ b/tests/unit/test_mangle.cpp @@ -0,0 +1,127 @@ +#include + +import std; +import mcpp.pm.mangle; + +using namespace mcpp::pm; + +TEST(Mangle, NameFormat) { + EXPECT_EQ(mangle_name("json", "1.2.3"), "json__v1_2_3__mcpp"); + EXPECT_EQ(mangle_name("mcpplibs.cmdline","2.0"),"mcpplibs.cmdline__v2_0__mcpp"); + EXPECT_EQ(mangle_name("a", "0"), "a__v0__mcpp"); +} + +TEST(Mangle, RewriteEmpty) { + std::map table; + EXPECT_EQ(rewrite_module_decls("", table), ""); + EXPECT_EQ(rewrite_module_decls("export module foo;\n", table), + "export module foo;\n") + << "empty rename table → no change"; +} + +TEST(Mangle, RewriteModuleDecl) { + std::map table { + {"json", "json__v2_0_0__mcpp"}, + }; + EXPECT_EQ(rewrite_module_decls("export module json;\n", table), + "export module json__v2_0_0__mcpp;\n"); + EXPECT_EQ(rewrite_module_decls("module json;\n", table), + "module json__v2_0_0__mcpp;\n"); +} + +TEST(Mangle, RewritePartitionDecl) { + std::map table { + {"json", "json__v2_0_0__mcpp"}, + }; + EXPECT_EQ(rewrite_module_decls("export module json:utils;\n", table), + "export module json__v2_0_0__mcpp:utils;\n"); + EXPECT_EQ(rewrite_module_decls("module json:impl;\n", table), + "module json__v2_0_0__mcpp:impl;\n"); +} + +TEST(Mangle, RewriteImports) { + std::map table { + {"json", "json__v2_0_0__mcpp"}, + }; + EXPECT_EQ(rewrite_module_decls("import json;\n", table), + "import json__v2_0_0__mcpp;\n"); + EXPECT_EQ(rewrite_module_decls("export import json;\n", table), + "export import json__v2_0_0__mcpp;\n"); + EXPECT_EQ(rewrite_module_decls("import json:utils;\n", table), + "import json__v2_0_0__mcpp:utils;\n"); +} + +TEST(Mangle, KeepBarePartitionImport) { + std::map table { + {"json", "json__v2_0_0__mcpp"}, + }; + // `import :P;` refers to the enclosing module's partition — not a + // named module; must NOT be rewritten. + EXPECT_EQ(rewrite_module_decls("import :utils;\n", table), + "import :utils;\n"); + EXPECT_EQ(rewrite_module_decls("module ;\n", table), + "module ;\n") + << "global module fragment opener has no name"; +} + +TEST(Mangle, KeepNonMatching) { + std::map table { + {"json", "json__v2_0_0__mcpp"}, + }; + // Other modules in the same file shouldn't be touched. + EXPECT_EQ(rewrite_module_decls("import std;\n", table), "import std;\n"); + EXPECT_EQ(rewrite_module_decls("import other;\n", table), "import other;\n"); + EXPECT_EQ(rewrite_module_decls("export module yaml;\n", table), + "export module yaml;\n"); +} + +TEST(Mangle, MultipleLines) { + std::map table { + {"json", "json__v2_0_0__mcpp"}, + }; + std::string in = + "// header comment\n" + "module;\n" + "#include \n" + "export module json;\n" + "import std;\n" + "import json:utils;\n" + "export int answer() { return 42; }\n"; + std::string expected = + "// header comment\n" + "module;\n" + "#include \n" + "export module json__v2_0_0__mcpp;\n" + "import std;\n" + "import json__v2_0_0__mcpp:utils;\n" + "export int answer() { return 42; }\n"; + EXPECT_EQ(rewrite_module_decls(in, table), expected); +} + +TEST(Mangle, LeadingWhitespace) { + std::map table { + {"json", "json__v2_0_0__mcpp"}, + }; + EXPECT_EQ(rewrite_module_decls(" export module json;\n", table), + " export module json__v2_0_0__mcpp;\n"); + EXPECT_EQ(rewrite_module_decls("\timport json;\n", table), + "\timport json__v2_0_0__mcpp;\n"); +} + +TEST(Mangle, NoTrailingNewline) { + std::map table { + {"json", "json__v2_0_0__mcpp"}, + }; + EXPECT_EQ(rewrite_module_decls("export module json;", table), + "export module json__v2_0_0__mcpp;"); +} + +TEST(Mangle, DottedNames) { + std::map table { + {"mcpplibs.cmdline", "mcpplibs.cmdline__v2_0_0__mcpp"}, + }; + EXPECT_EQ(rewrite_module_decls("export module mcpplibs.cmdline;\n", table), + "export module mcpplibs.cmdline__v2_0_0__mcpp;\n"); + EXPECT_EQ(rewrite_module_decls("import mcpplibs.cmdline;\n", table), + "import mcpplibs.cmdline__v2_0_0__mcpp;\n"); +}