Skip to content

Add vcpkg NuGet binary cache to CI#58

Merged
huangminghuang merged 10 commits into
masterfrom
codex/vcpkg-nuget-binary-cache
May 19, 2026
Merged

Add vcpkg NuGet binary cache to CI#58
huangminghuang merged 10 commits into
masterfrom
codex/vcpkg-nuget-binary-cache

Conversation

@huangminghuang
Copy link
Copy Markdown
Contributor

@huangminghuang huangminghuang commented May 18, 2026

Why

vcpkg dependencies, especially LLVM, are expensive to build from source and make CI slow whenever the existing archive cache misses. This PR moves compiled vcpkg dependency reuse to a durable NuGet-backed binary cache in GitHub Packages.

The goal is to make repeated CI and local builds faster and more predictable while keeping the existing containerized CI environment as the source of ABI stability. The local helper script now derives its expected Ubuntu/LLVM environment from the workflow-selected CI Dockerfile, so developer builds can validate that they are aligned before expecting NuGet cache reuse.

Summary

  • enable vcpkg binary caching through GitHub Packages/NuGet for trusted CI runs
  • keep fork PRs on vcpkg's local/default binary cache path to avoid package credential exposure
  • add scripts/build-with-github-vcpkg-cache.sh as the shared local/CI build entrypoint
  • switch the workflow build/test job to call the helper script directly
  • support intent-based helper modes: developer, trusted-ci, and forked-pr-ci
  • have the helper discover the CI platform Dockerfile through .github/workflows/build.yaml and the configured platform file
  • validate local Ubuntu codename, architecture, Clang/LLVM major version, apt.llvm.org package family, triplets, GitHub package auth, and NuGet source setup before building
  • enable ccache explicitly when available and propagate the CMake compiler launcher into ExternalProject sub-builds
  • add Mono to the Ubuntu 24 builder image so vcpkg's NuGet client can run in Linux containers
  • update Docker actions to Node 24-compatible versions
  • switch the Ubuntu 24 builder from the Jammy LLVM apt repo to the Noble LLVM 18 apt repo
  • remove ccache as a vcpkg port compiler launcher now that compiled vcpkg packages are restored from NuGet
  • install bundled LLVM host tools as copied executables instead of build-tree symlinks so CDT installs are self-contained
  • update BUILD.md with the current build flow, helper script modes, optional NuGet binary cache setup, and installed tool inventory

Validation

CI:

  • Draft PR CI passed after rebuilding the Ubuntu 24 builder image
  • CI populated the new Noble-based vcpkg NuGet cache
  • CI logs show vcpkg uploading packages to GitHub Packages/NuGet

Local:

  • Refreshed local GitHub auth with read:packages
  • Aligned local Clang with CI by using apt.llvm.org/noble llvm-toolchain-noble-18
  • Confirmed local vcpkg cache restore:
Restored 9 package(s) from NuGet
  • Verified helper script syntax and whitespace:
bash -n scripts/build-with-github-vcpkg-cache.sh
git diff --check
  • Tested developer-mode helper build using the existing NuGet build directory:
scripts/build-with-github-vcpkg-cache.sh --build-dir build-local-nuget-cache
  • Confirmed local tests passed:
100% tests passed, 0 tests failed out of 23
  • Confirmed ccache is enabled by the helper when available and CMake launcher settings are propagated into the root, libraries, tests, and toolchain-tester build directories

Install rule:

  • Tested DESTDIR=/tmp/wire-cdt-install-test cmake --install build-local-nuget-cache
  • Confirmed installed LLVM tools in usr/local/cdt/bin are regular ELF executables, not symlinks back to build-local-nuget-cache/vcpkg_installed

Notes

This intentionally keeps the existing containerized CI build. The container stabilizes the compiler/toolchain ABI that vcpkg uses for binary cache keys.

For local cache hits, developers should use the developer mode helper path documented in BUILD.md. Trusted CI uses trusted-ci mode with GitHub Packages read/write access; fork PR CI uses forked-pr-ci mode without package credentials.

@huangminghuang huangminghuang marked this pull request as ready for review May 19, 2026 14:54
@huangminghuang huangminghuang requested a review from heifner May 19, 2026 14:55
Copy link
Copy Markdown
Contributor

@heifner heifner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and tested this branch locally — overall looks good, and the NuGet binary-cache path works end-to-end (restored 9 packages incl. llvm@18.1.6 from GitHub Packages in ~1.2 min, full build + 23/23 ctest pass, install tree contains real ELF tools rather than build-dir symlinks).

Two non-blocking build/robustness notes inline; neither blocks merge.

Comment thread tools/CMakeLists.txt Outdated
Comment thread scripts/build-with-github-vcpkg-cache.sh Outdated
@huangminghuang huangminghuang requested a review from heifner May 19, 2026 18:28
@huangminghuang huangminghuang merged commit 0ea07e6 into master May 19, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants