Skip to content

feat(deps): multi-version mangling — Level 1 fallback #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Sunrisepeak merged 1 commit into from
May 9, 2026
Merged

feat(deps): multi-version mangling — Level 1 fallback #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 24 additions & 6 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@

## [Unreleased] — 0.0.3

依赖解析体系的两步演进:0.0.2 release tag 之后合入的 transitive walker
之上,这一版叠加 SemVer 合并;后续 PR 还会补上多版本 mangling 兜底。
依赖解析体系的三步演进:0.0.2 release tag 之后合入 transitive walker,
这一版补齐 SemVer 合并(Level 2)+ 多版本 mangling 兜底(Level 1)。

### 新增

Expand All @@ -20,10 +20,28 @@
以不同版本约束声明时,resolver 会把两条原始约束 AND 合并(裸版本号视作
`=X.Y.Z`),向 index 重新查询,选出同时满足两侧的具体版本。若该版本与
此前已 pin 的不一致,旧的 manifest 与 `[build].include_dirs` 会被原地
替换为新版本的内容,孩子依赖也按新 manifest 重新入队。完全无重叠
(典型如 `=0.0.1` 对 `=0.0.2`)仍硬报错并提示后续 PR 会用多版本
mangling 兜底。新增 e2e `32_semver_merge.sh` 覆盖兼容合并 + 不可调和
两条主链路。
替换为新版本的内容,孩子依赖也按新 manifest 重新入队。新增 e2e
`32_semver_merge.sh` 覆盖兼容合并 + 不可调和两条主链路。

- ✅ **多版本 mangling 兜底(Level 1)** —— SemVer 合并失败时(典型如
`=0.0.1` ⨯ `=0.0.2` 这种无重叠的 pin),resolver 不再硬报错,而是把次要
版本的源码 stage 到 `target/.mangled/<consumer>/...` 下,通过正则改写
`(export )?module X;` / `(export )?module X:Y;` / `(export )?import X;`
把模块名替换成 `<X>__v<M>_<m>_<p>__mcpp` 形式,让两个 BMI 在同一构建图
里以不同模块名共存(C++23 module attachment 帮我们做 ABI 隔离,无需额外
namespace mangle)。直接 consumer 的源码也一并 stage + 改写,让它的
`import` 指向 mangled 副本。MVP 范围:仅处理 dep-as-consumer + 叶子
secondary 两种情形,主包做 consumer 或 secondary 还有自己的 transitive
deps 时报清晰错误并建议显式 pin。新增 `src/pm/mangle.cppm`(纯改写
helper + 11 个单元测试)和 e2e `33_multi_version_mangling.sh`。

### 改进

- 🔧 **构建后端按需为多包做 obj 路径命名空间** —— `plan.cppm` 检测到
跨包同名源文件(多版本 mangling 后两个 `parse.cppm` 同时存在的常见情形)
时,自动把 `obj/<file>.o` 改为 `obj/<sanitized-pkg>/<file>.o`,`.ddi`
扫描产物随之放在 object 同目录下。无碰撞时仍是原始 `obj/<file>.o`
布局,不影响现有缓存命中。

第二个公开版本。新增 C 语言一等公民支持、xpkg 风格依赖命名空间、包管理子系统骨架重构,以及 lib-root 约定。

Expand Down
8 changes: 6 additions & 2 deletions src/build/ninja_backend.cppm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -357,15 +357,19 @@ std::string emit_ninja_string(const BuildPlan& plan) {

if (dyndep) {
// ── Phase 1: scan edges (one .ddi per TU). ──────────────────────
// .ddi is placed beside the object: obj/<src>.ddi.
// .ddi is placed beside the object so multi-version mangling can
// namespace by package without producing two `build` rules with
// the same `.ddi` output (plan.cppm switches `cu.object` from
// `obj/<file>.o` to `obj/<pkg>/<file>.o` whenever a basename
// collides across packages — `.ddi` follows that placement).
// Skip .c files: they have no `import`s and don't need P1689 scan;
// running them through cxx_scan would route them through g++ /
// -fmodules which is exactly what C support is here to avoid.
std::vector<std::string> ddi_paths;
ddi_paths.reserve(plan.compileUnits.size());
for (auto& cu : plan.compileUnits) {
if (is_c_source(cu.source)) continue;
auto ddi = (std::filesystem::path("obj")
auto ddi = (cu.object.parent_path()
/ cu.source.filename()).string() + ".ddi";
ddi_paths.push_back(ddi);
append(std::format("build {} : cxx_scan {}\n",
Expand Down
23 changes: 22 additions & 1 deletion src/build/plan.cppm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,12 +96,33 @@ BuildPlan make_plan(const mcpp::manifest::Manifest& manifest,
plan.stdBmiPath = stdBmiPath;
plan.stdObjectPath = stdObjectPath;

// 1a. Detect basename collisions across packages (multi-version mangling
// stages a second copy of the same dep, so `parse.cppm` and friends
// can show up twice). For colliding files we namespace the object
// path by the unit's owning package so `obj/<file>.o` doesn't get
// two `build` rules.
std::map<std::string, int> basenameCount;
for (auto idx : topoOrder) {
basenameCount[object_filename_for(graph.units[idx].path)]++;
}
auto sanitize_pkg = [](const std::string& s) {
std::string out; out.reserve(s.size());
for (char c : s) out += (c == '.' ? '_' : c);
return out;
};

// 1. Compile units in topological order
for (auto idx : topoOrder) {
auto& u = graph.units[idx];
CompileUnit cu;
cu.source = u.path;
cu.object = std::filesystem::path("obj") / object_filename_for(u.path);
const auto fname = object_filename_for(u.path);
if (basenameCount[fname] > 1 && !u.packageName.empty()) {
cu.object = std::filesystem::path("obj")
/ sanitize_pkg(u.packageName) / fname;
} else {
cu.object = std::filesystem::path("obj") / fname;
}
if (u.provides) {
cu.providesModule = u.provides->logicalName;
}
Expand Down
209 changes: 190 additions & 19 deletions src/cli.cppm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import mcpp.config;
import mcpp.fetcher;
import mcpp.pm.resolver; // PR-R4: extracted from cli.cppm
import mcpp.pm.commands; // PR-R5: cmd_add / cmd_remove / cmd_update live here now
import mcpp.pm.mangle; // Level 1 multi-version fallback (cross-major coexistence)
import mcpp.ui;
import mcpp.bmi_cache;
import mcpp.dyndep;
Expand Down Expand Up @@ -1062,11 +1063,15 @@ prepare_build(bool print_fingerprint,
};
std::map<ResolvedKey, ResolvedRecord> resolved;

// Sentinel for "the consumer is the main package" (no dep_manifests entry).
constexpr std::size_t kMainConsumer = static_cast<std::size_t>(-1);

struct WorkItem {
std::string name; // dep map key as written
mcpp::manifest::DependencySpec spec; // copy (we may mutate version)
std::string requestedBy; // who asked for it
std::string originalConstraint; // spec.version BEFORE pinning (for SemVer merge)
std::size_t consumerDepIndex; // dep_manifests slot of who pushed this child; kMainConsumer for main
};
std::deque<WorkItem> worklist;

Expand Down Expand Up @@ -1212,15 +1217,76 @@ prepare_build(bool print_fingerprint,
}
};

// Stage a dep's source files into a fresh directory, rewriting their
// module / import declarations against `rename`. Used by the multi-
// version mangling fallback (Level 1) so two cross-major copies of
// the same package can coexist with distinct module names.
//
// Headers (referenced via `[build].include_dirs`) are NOT staged —
// those keep pointing at the original install dir via absolutized
// include paths.
auto stage_with_rewrite = [](const std::filesystem::path& srcRoot,
const std::filesystem::path& dstRoot,
const mcpp::manifest::Manifest& depManifest,
const std::map<std::string, std::string>& rename)
-> std::expected<void, std::string>
{
std::error_code ec;
std::filesystem::create_directories(dstRoot, ec);
if (ec) return std::unexpected(std::format(
"stage: cannot create '{}': {}", dstRoot.string(), ec.message()));

// Resolve the source globs against the original root, falling
// back to the convention default if the manifest didn't set any.
std::vector<std::string> globs = depManifest.modules.sources;
if (globs.empty()) {
globs = { "src/**/*.cppm", "src/**/*.cpp",
"src/**/*.cc", "src/**/*.c" };
}
std::set<std::filesystem::path> sourceFiles;
for (auto const& g : globs) {
for (auto& p : mcpp::modgraph::expand_glob(srcRoot, g)) {
sourceFiles.insert(p);
}
}
if (sourceFiles.empty()) {
return std::unexpected(std::format(
"stage: no source files found under '{}' (globs={})",
srcRoot.string(), globs.size()));
}

for (auto const& f : sourceFiles) {
auto rel = std::filesystem::relative(f, srcRoot, ec);
if (ec) return std::unexpected(std::format(
"stage: cannot relativize '{}': {}", f.string(), ec.message()));
auto dst = dstRoot / rel;
std::filesystem::create_directories(dst.parent_path(), ec);

std::ifstream is(f);
if (!is) return std::unexpected(std::format(
"stage: cannot read '{}'", f.string()));
std::stringstream buf; buf << is.rdbuf();
std::string content = buf.str();

std::string out = mcpp::pm::rewrite_module_decls(content, rename);
std::ofstream os(dst);
if (!os) return std::unexpected(std::format(
"stage: cannot write '{}'", dst.string()));
os << out;
}
return {};
};

// Seed the worklist from the main manifest. Dev-deps only when the
// caller wants them; they're never propagated transitively.
const std::string mainPkgLabel = m->package.name;
for (auto& [n, s] : m->dependencies) {
worklist.push_back({n, s, mainPkgLabel, s.version});
worklist.push_back({n, s, mainPkgLabel, s.version, kMainConsumer});
}
if (includeDevDeps) {
for (auto& [n, s] : m->devDependencies) {
worklist.push_back({n, s, mainPkgLabel + " (dev-dep)", s.version});
worklist.push_back({n, s, mainPkgLabel + " (dev-dep)",
s.version, kMainConsumer});
}
}

Expand Down Expand Up @@ -1274,21 +1340,124 @@ prepare_build(bool print_fingerprint,
item.originalConstraint,
fetcher);
if (!merged) {
return std::unexpected(std::format(
"dependency '{}{}{}' has irreconcilable versions in "
"the transitive graph:\n"
" '{}' (constraint '{}') requested by '{}'\n"
" '{}' (constraint '{}') requested by '{}'\n"
"SemVer merge: {}\n"
"C++ modules require a single global version of each "
"package; pick a version compatible with both "
"consumers, or ask one upstream to widen its dep "
"range. (cross-major fallback via multi-version "
"mangling is planned in a follow-up PR)",
key.ns, key.ns.empty() ? "" : ".", key.shortName,
it->second.version, it->second.constraint, it->second.requestedBy,
spec.version, item.originalConstraint, item.requestedBy,
merged.error()));
// Level 1 fallback: multi-version mangling. Two
// versions can't be reconciled by SemVer, but they
// can coexist in the same build if we mangle the
// secondary copy's module name and rewrite the one
// consumer that asked for it. The primary keeps its
// authored module name so consumers that don't care
// about the secondary see no churn.
//
// MVP scope (these limits surface as clear errors):
// * The conflicting consumer must be a dep, not
// the main package — main-package mangling
// would mean rewriting user-authored sources,
// which is too surprising for a fallback path.
// * The secondary version must be a leaf (no own
// transitive deps) — recursive mangling is
// deferred to a follow-up.
if (item.consumerDepIndex == kMainConsumer) {
return std::unexpected(std::format(
"dependency '{}{}{}' has irreconcilable versions:\n"
" '{}' (constraint '{}') requested by '{}'\n"
" '{}' (constraint '{}') requested by '{}'\n"
"SemVer merge: {}\n"
"Multi-version mangling can't help here — the conflict "
"involves the main package directly. Pin one version "
"explicitly in your mcpp.toml.",
key.ns, key.ns.empty() ? "" : ".", key.shortName,
it->second.version, it->second.constraint, it->second.requestedBy,
spec.version, item.originalConstraint, item.requestedBy,
merged.error()));
}

auto loaded = loadVersionDep(name, spec.version);
if (!loaded) return std::unexpected(loaded.error());
auto& [secondaryRoot, secondaryManifest] = *loaded;

if (!secondaryManifest.dependencies.empty()) {
return std::unexpected(std::format(
"dependency '{}{}{}' has irreconcilable versions:\n"
" '{}' requested by '{}'\n"
" '{}' requested by '{}'\n"
"Multi-version mangling fallback only handles leaf "
"secondaries in 0.0.3 — but the secondary v{} declares "
"its own dependencies, which would need recursive "
"mangling. Pin one version explicitly, or wait for "
"the recursive-mangling extension.",
key.ns, key.ns.empty() ? "" : ".", key.shortName,
it->second.version, it->second.requestedBy,
spec.version, item.requestedBy,
spec.version));
}

// Module names in the source files use the dep's full
// [package].name (e.g. "mcpplibs.cmdline"), not the
// namespaced-subtable shortName. Use that for the
// rename key so the rewriter actually matches what the
// .cppm sources declare.
const std::string moduleName = secondaryManifest.package.name;
std::string mangled =
mcpp::pm::mangle_name(moduleName, spec.version);

// Stage layout:
// <root>/target/.mangled/<consumerPkg>/<dep>__<version>/ ← rewritten secondary source
// <root>/target/.mangled/<consumerPkg>/__self__/ ← rewritten consumer source
auto& consumerManifest = *dep_manifests[item.consumerDepIndex];
auto consumerRoot = packages[item.consumerDepIndex + 1].root;
auto stageBase = *root / "target" / ".mangled"
/ consumerManifest.package.name;
auto secStage = stageBase
/ std::format("{}__{}", moduleName, spec.version);
auto consumerStage = stageBase / "__self__";

std::map<std::string, std::string> rename{ {moduleName, mangled} };
if (auto r = stage_with_rewrite(secondaryRoot, secStage,
secondaryManifest, rename); !r)
return std::unexpected(r.error());
if (auto r = stage_with_rewrite(consumerRoot, consumerStage,
consumerManifest, rename); !r)
return std::unexpected(r.error());

// Re-anchor the consumer's PackageRoot at its staged copy
// so the modgraph scanner picks up the rewritten imports.
packages[item.consumerDepIndex + 1].root = consumerStage;

// Record the staged secondary as a brand-new dep entry
// under its mangled name, so future encounters of this
// exact (ns, mangled) pair dedup cleanly. The original
// primary entry (it->second) is untouched.
auto stagedManifest = secondaryManifest;
// Update [package].name to the mangled module name so
// the modgraph validator (which checks "exported module
// must be prefixed by package name") accepts the
// rewritten sources.
stagedManifest.package.name = mangled;
// Absolutize secondary's include_dirs against its original
// install root so the staged copy still finds headers.
for (auto& inc : stagedManifest.buildConfig.includeDirs) {
if (inc.is_relative()) inc = secondaryRoot / inc;
}

dep_manifests.push_back(
std::make_unique<mcpp::manifest::Manifest>(std::move(stagedManifest)));
packages.push_back({secStage, *dep_manifests.back()});
auto added = propagateIncludeDirs(secStage, *dep_manifests.back());

ResolvedKey mangledKey{key.ns, mangled};
resolved[mangledKey] = ResolvedRecord{
.version = spec.version,
.constraint = item.originalConstraint,
.requestedBy = item.requestedBy,
.source = "version",
.depIndex = dep_manifests.size() - 1,
.includeDirsAdded = std::move(added),
};

mcpp::ui::info("Mangled",
std::format("{} v{} ↔ v{} → {} (cross-major fallback)",
moduleName, it->second.version, spec.version, mangled));
continue;
}

// Combine the constraint strings so future merges AND with
Expand Down Expand Up @@ -1365,7 +1534,8 @@ prepare_build(bool print_fingerprint,
for (auto& [child_name, child_spec] :
dep_manifests[it->second.depIndex]->dependencies) {
worklist.push_back({child_name, child_spec, newLabel,
child_spec.version});
child_spec.version,
it->second.depIndex});
}
continue;
}
Expand Down Expand Up @@ -1514,9 +1684,10 @@ prepare_build(bool print_fingerprint,
key.ns.empty() ? "" : ".",
key.shortName,
sourceKind == "version" ? spec.version : sourceKind);
const std::size_t selfIdx = dep_manifests.size() - 1;
for (auto& [child_name, child_spec] : dep_manifests.back()->dependencies) {
worklist.push_back({child_name, child_spec, thisDepLabel,
child_spec.version});
child_spec.version, selfIdx});
}
}

Expand Down
Loading
Loading