build: replace ccache + GHA cache with sccache backed by Azure Blob Storage#208
Merged
Conversation
Contributor
|
xref - should close out: #166 |
bleggett
previously approved these changes
Jul 14, 2026
…torage The previous compile cache serialized the whole ccache directory into a single GitHub Actions cache blob per flavor/arch. Those blobs shared the repo-wide 10GB cache budget and were routinely evicted before any build could benefit from them, and the 7-day per-entry TTL needed a scheduled workflow (cache-refresh.yml) whose only job was to keep entries alive. Switch the compile phase to sccache with Azure Blob Storage as a shared backend (container `sccache` on the `ederasccache` storage account). Cache objects are stored and fetched individually, so there is no more monolithic save/restore step, no size cap to prune against, and all builders across all flavors, arches, and runs read and write the same namespace concurrently. Garbage collection is handled service-side by an Azure lifecycle rule that deletes objects unaccessed for 30 days, which lets cache-refresh.yml be deleted outright. Details: - The build environment now comes from the published kernel-buildenv image, pinned by digest (see Dockerfile.buildenv and buildenv.yml). That image carries the pinned, checksum-verified sccache release and the /usr/lib/sccache compiler wrapper scripts, so the Dockerfile's inline toolchain construction goes away entirely and the compilers can only change through a reviewed digest bump - which matters now that cache keys include the compiler binary's hash. - The generated docker.sh passes the SCCACHE_AZURE_* environment through by name only, so the script (which is uploaded as a workflow artifact) never contains credentials. - CI selects credentials from an explicit allowlist of maintainer-gated triggers: push, schedule, and workflow_dispatch get the read-write connection string, and everything else - pull requests today, any trigger added in the future - fails closed to a read-only token, so unreviewed code can never poison cache objects later consumed by release builds. Fork PRs (no secrets) fall back to a bind-mounted local disk cache, the same fallback that keeps local developer builds cached across runs. - docker-build-internal.sh prints `sccache --show-stats` after each compile so hit rates and backend errors are visible in build logs. - The Dockerfile packaging targets and build context are renamed ccachebuild -> prebuilt, since what they consume are artifacts compiled out-of-band in the docker run phase, not anything ccache-specific. Signed-off-by: Steven Noonan <steven@edera.dev>
…stings rsync.kernel.org caps concurrent connections (currently 100) and refuses new sessions with exit code 5 when saturated, and matrix generation made one rsync connection for the firmware listing plus one per kernel major series - a connection-slot lottery on every run. Worse, rsync's stderr was discarded, so CI logs never showed the "@error: max connections" message; the failure had to be reproduced locally to be diagnosed. Enumerate releases from git tags instead: `git ls-remote --tags --refs` against git.kernel.org fetches only the server-side-filtered tag advertisement (no clone), and git.kernel.org does not impose the rsync daemon's aggressive connection cap. Release tags map 1:1 to published artifacts: vX.Y[.Z] in stable/linux.git to linux-X.Y[.Z].tar.xz, and YYYYMMDD in firmware/linux-firmware.git to linux-firmware-YYYYMMDD.tar.xz (the live tag sets were verified to match the rsync directory listings exactly, including the firmware range 20190312..present). The listing helper retries with exponential backoff and jitter, and prints the underlying stderr on each failed attempt, so any residual network failure is both tolerated and visible in CI logs. One behavioral note: linux.git tags only reach back to v2.6.11 while the rsync tree lists tarballs to v1.0; nothing anywhere near that old is buildable from this repo. Signed-off-by: Steven Noonan <steven@edera.dev>
tycho
force-pushed
the
steven/azure-sccache
branch
from
bd7c21c to
f15fdc2
Compare
kaniini
approved these changes
Jul 15, 2026
azenla
approved these changes
Jul 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The previous compile cache serialized the whole ccache directory into a single GitHub Actions cache blob per flavor/arch. Those blobs shared the repo-wide 10GB cache budget and were routinely evicted before any build could benefit from them, and the 7-day per-entry TTL needed a scheduled workflow (cache-refresh.yml) whose only job was to keep entries alive.
Switch the compile phase to sccache with Azure Blob Storage as a shared backend (container
sccacheon theederasccachestorage account). Cache objects are stored and fetched individually, so there is no more monolithic save/restore step, no size cap to prune against, and all builders across all flavors, arches, and runs read and write the same namespace concurrently. Garbage collection is handled service-side by an Azure lifecycle rule that deletes objects unaccessed for 30 days, which lets cache-refresh.yml be deleted outright.Details:
sccache --show-statsafter each compile so hit rates and backend errors are visible in build logs.