Skip to content

ci(manual-build): bring disk-cleanup fixes onto staging#719

Merged
yahortsaryk merged 1 commit into
stagingfrom
ci/staging-disk-cleanup
May 29, 2026
Merged

ci(manual-build): bring disk-cleanup fixes onto staging#719
yahortsaryk merged 1 commit into
stagingfrom
ci/staging-disk-cleanup

Conversation

@yahortsaryk
Copy link
Copy Markdown
Contributor

Summary

blockchain-node#714 (release/80009 → staging) merged at the first version of the manual-build workflow's disk-cleanup step, which turned out to be a no-op. The fixes that actually reclaim disk landed on release/80009 later as five iterative commits — those are now orphaned because the PR was already closed.

This PR brings the final version of .github/workflows/manual-build.yaml directly to staging, squashed into one commit.

What the Reclaim step now does

  • Prints docker system df and the list of buildx_buildkit_* volumes before AND after — diagnostic visibility for future failures
  • docker container prune --force — drops stopped containers (this turned out to be the real disk hog: 36 GiB freed on a runner at 86% usage)
  • docker image prune --force — drops dangling untagged images
  • Removes orphaned buildkit state volumes from prior builder instances
  • Iterates every still-existing buildx builder and caps its cache at 30 GiB

Each command is || true so the step never blocks the build on an edge case.

Observed result

On a fresh build from release/80009: 124 GiB → 88 GiB used (reclaimed 36 GiB), build then completed successfully.

Test plan

  • Next manual build from staging: "Reclaim disk on self-hosted runner" step prints real docker system df and the post-prune df -h / shows reclaimed disk

The release/80009 → staging PR (#714) merged before the CI-side
disk-cleanup fixes were ready. Subsequent commits to release/80009
fixing the 'No space left on device' on the self-hosted runner are
orphaned (the PR is already closed). This PR brings the final state
of those fixes directly to staging.

Reclaim step now:
* prints docker system df + buildx_buildkit_* volume list before
  and after, so future failures show exactly where disk is going
* drops stopped containers + dangling images
* removes orphaned buildx_buildkit_* state volumes from prior
  builder instances
* iterates every still-existing buildx builder and caps each one's
  cache at 30 GiB

Observed empirically on dev: reclaimed 36 GiB on a runner sitting
at 86% used (mostly from leaked stopped containers carrying
overlay writable layers).
@yahortsaryk yahortsaryk merged commit 3c7f71b into staging May 29, 2026
9 of 11 checks passed
@yahortsaryk yahortsaryk deleted the ci/staging-disk-cleanup branch May 29, 2026 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant