-
Notifications
You must be signed in to change notification settings - Fork 0
feat: S3-backed cache-s3 siblings (golang/rust/node-pnpm) for Hetzner RGW #187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: v4-beta
Are you sure you want to change the base?
Changes from all commits
4a5c0ed
9cd6d60
38ef53c
7b318f1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| name: Setup S3-backed cache for a Golang application (Hetzner RGW) | ||
| author: 'MiLaboratories' | ||
| description: | | ||
| S3-backed sibling of golang/cache. Restores/saves the Go module + build | ||
| caches to an S3 endpoint (the Hetzner RadosGW at s3.hz.platforma.bio) via | ||
| tespkg/actions-cache, instead of the GitHub-hosted cache. | ||
|
|
||
| Use on the hz self-hosted runners (hz-rl8-*, hz-ubuntu-dind): the cache is | ||
| restored per-job and saved on success, so there is no shared-hostPath cache | ||
| on the node and therefore no cross-job contamination. Linux only (the hz | ||
| fleet is Linux/x86); macOS/Windows jobs keep using golang/cache. | ||
|
|
||
| The cache key scheme is identical to golang/cache, so switching backends | ||
| does not change keys. | ||
|
|
||
| inputs: | ||
| cache-version: | ||
| description: | | ||
| Change this value to 'reset' the cache for a job (forces a cold build). | ||
| required: false | ||
| default: 'v1' | ||
| cache-dependency-hashfiles-path: | ||
| description: | | ||
| hashFiles() pattern that produces the cache key suffix. | ||
| required: false | ||
| default: '**/go.work.sum' | ||
|
|
||
| # --- S3 / RadosGW --- | ||
| endpoint: | ||
| description: "S3 endpoint host, no scheme (split-DNS: resolves to the in-cluster RGW on hz, public elsewhere)." | ||
| required: false | ||
| default: 's3.hz.platforma.bio' | ||
| bucket: | ||
| description: "S3 bucket." | ||
| required: false | ||
| default: 'ci-actions-cache' | ||
| region: | ||
| description: "S3 region." | ||
| required: false | ||
| default: 'us-east-1' | ||
| insecure: | ||
| description: "Use plain HTTP instead of TLS to the endpoint." | ||
| required: false | ||
| default: 'false' | ||
| access-key: | ||
| description: "S3 access key (pass from the org secret HZ_CI_CACHE_S3_ACCESS_KEY)." | ||
| required: true | ||
| secret-key: | ||
| description: "S3 secret key (pass from the org secret HZ_CI_CACHE_S3_SECRET_KEY)." | ||
| required: true | ||
| use-fallback: | ||
| description: "Fall back to the GitHub-hosted cache if the S3 endpoint is unreachable." | ||
| required: false | ||
| default: 'true' | ||
|
|
||
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - name: Cache Golang modules in S3 (RGW) | ||
| uses: tespkg/actions-cache@e07e2d4953dc8c020d447363e5064e36d04f3cf9 # v1.10.2 | ||
| with: | ||
| endpoint: ${{ inputs.endpoint }} | ||
| region: ${{ inputs.region }} | ||
| bucket: ${{ inputs.bucket }} | ||
| insecure: ${{ inputs.insecure }} | ||
| accessKey: ${{ inputs.access-key }} | ||
| secretKey: ${{ inputs.secret-key }} | ||
| use-fallback: ${{ inputs.use-fallback }} | ||
| retry: 'true' | ||
| retry-count: '3' | ||
| # Only the build cache — NOT ~/go/pkg/mod. The module cache is read-only | ||
| # (0444) and tespkg/actions-cache extracts with `tar --keep-old-files`, | ||
| # which fails ("Cannot open: File exists") on the module cache; the build | ||
| # cache is the real compile-time win, and modules repopulate from | ||
| # GOPROXY (use a cached-only proxy). (Stock actions/cache overwrote, so | ||
| # this only bites the S3-backed action.) | ||
| path: | | ||
| ~/.cache/go-build | ||
| key: ${{ runner.os }}-${{ runner.arch }}-cache-go-${{ inputs.cache-version }}-${{ hashFiles(inputs.cache-dependency-hashfiles-path) }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-${{ runner.arch }}-cache-go-${{ inputs.cache-version }}- | ||
|
Comment on lines
+60
to
+81
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The sibling Prompt To Fix With AIThis is a comment left during a code review.
Path: actions/golang/cache-s3/action.yaml
Line: 60-76
Comment:
**No `save-always` equivalent; cache not saved on job failure**
The sibling `golang/cache` defaults `cache-save-always: true`, which passes `save-always: true` to `actions/cache` so the cache is written even when a later step fails. `tespkg/actions-cache` does not document a `save-always` input, so the S3 action only saves on clean-run success. On the hz fleet, where jobs are more likely to see flaky infra failures, this means a partially-built Go module cache is silently discarded and the next run has to start cold. Consider documenting this divergence in the action description, or adding a separate post-step / `save-always`-like mechanism if the action supports it.
How can I resolve this? If you propose a fix, please make it concise. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| name: Cache NodeJS PNPM in S3 (Hetzner RGW) | ||
| author: 'MiLaboratories' | ||
| description: | | ||
| S3-backed sibling of node/cache-pnpm. Restores/saves the pnpm content- | ||
| addressable store to the Hetzner RadosGW (s3.hz.platforma.bio) via | ||
| tespkg/actions-cache, instead of the GitHub-hosted cache. Use on the hz | ||
| self-hosted runners. Linux only. | ||
|
|
||
| The pnpm store path is resolved dynamically (`pnpm store path`), so it works | ||
| regardless of where the runner image places the store. The cache key scheme | ||
| is identical to node/cache-pnpm. | ||
|
|
||
| inputs: | ||
| cache-version: | ||
| description: "Change this value to 'reset' the cache for a job." | ||
| required: false | ||
| default: 'v1' | ||
| cache-hashfiles-search-path: | ||
| description: "hashFiles() pattern for pnpm-lock.yaml." | ||
| required: false | ||
| default: '**/pnpm-lock.yaml' | ||
|
|
||
| # --- S3 / RadosGW --- | ||
| endpoint: | ||
| description: "S3 endpoint host, no scheme." | ||
| required: false | ||
| default: 's3.hz.platforma.bio' | ||
| bucket: | ||
| description: "S3 bucket." | ||
| required: false | ||
| default: 'ci-actions-cache' | ||
| region: | ||
| description: "S3 region." | ||
| required: false | ||
| default: 'us-east-1' | ||
| insecure: | ||
| description: "Use plain HTTP instead of TLS to the endpoint." | ||
| required: false | ||
| default: 'false' | ||
| access-key: | ||
| description: "S3 access key (pass from the org secret HZ_CI_CACHE_S3_ACCESS_KEY)." | ||
| required: true | ||
| secret-key: | ||
| description: "S3 secret key (pass from the org secret HZ_CI_CACHE_S3_SECRET_KEY)." | ||
| required: true | ||
| use-fallback: | ||
| description: "Fall back to the GitHub-hosted cache if the S3 endpoint is unreachable." | ||
| required: false | ||
| default: 'true' | ||
|
|
||
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - name: Get pnpm store path | ||
| id: pnpm-store-dir | ||
| shell: bash | ||
| run: echo "dir=$(pnpm store path)" >> ${GITHUB_OUTPUT} | ||
|
|
||
| - name: Cache Node modules in S3 (RGW) | ||
| uses: tespkg/actions-cache@e07e2d4953dc8c020d447363e5064e36d04f3cf9 # v1.10.2 | ||
| with: | ||
| endpoint: ${{ inputs.endpoint }} | ||
| region: ${{ inputs.region }} | ||
| bucket: ${{ inputs.bucket }} | ||
| insecure: ${{ inputs.insecure }} | ||
| accessKey: ${{ inputs.access-key }} | ||
| secretKey: ${{ inputs.secret-key }} | ||
| use-fallback: ${{ inputs.use-fallback }} | ||
| retry: 'true' | ||
| retry-count: '3' | ||
| path: | | ||
| ${{ steps.pnpm-store-dir.outputs.dir }} | ||
| ~/.pnpm-store | ||
| key: ${{ runner.os }}-${{ runner.arch }}-cache-pnpm-${{ inputs.cache-version }}-genericnodejs-${{ hashFiles(inputs.cache-hashfiles-search-path) }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-${{ runner.arch }}-cache-pnpm-${{ inputs.cache-version }}-genericnodejs- |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| name: Setup S3-backed cache for a Rust application (Hetzner RGW) | ||
| author: 'MiLaboratories' | ||
| description: | | ||
| S3-backed sibling of rust/cache. Restores/saves the Cargo registry/git caches | ||
| and the target/ dir to the Hetzner RadosGW (s3.hz.platforma.bio) via | ||
| tespkg/actions-cache, instead of the GitHub-hosted cache. Use on the hz | ||
| self-hosted runners. Linux only. | ||
|
|
||
| NOTE: the hz-rl8 runner image bakes CARGO_HOME=/opt/rust/cargo (the rustup | ||
| install lives there), so the Cargo registry cache lands under that path, not | ||
| ~/.cargo. Pass cargo-home accordingly when consuming on rl8. | ||
|
|
||
| The cache key scheme is identical to rust/cache. | ||
|
|
||
| inputs: | ||
| cache-version: | ||
| description: "Change this value to 'reset' the cache for a job." | ||
| required: false | ||
| default: 'v1' | ||
| cache-hashfiles-search-path: | ||
| description: "hashFiles() pattern for Cargo.lock." | ||
| required: false | ||
| default: '**/Cargo.lock' | ||
| cargo-home: | ||
| description: "CARGO_HOME path (hz-rl8 image bakes /opt/rust/cargo; default assumes ~/.cargo)." | ||
| required: false | ||
| default: '~/.cargo' | ||
|
|
||
| # --- S3 / RadosGW --- | ||
| endpoint: | ||
| description: "S3 endpoint host, no scheme." | ||
| required: false | ||
| default: 's3.hz.platforma.bio' | ||
| bucket: | ||
| description: "S3 bucket." | ||
| required: false | ||
| default: 'ci-actions-cache' | ||
| region: | ||
| description: "S3 region." | ||
| required: false | ||
| default: 'us-east-1' | ||
| insecure: | ||
| description: "Use plain HTTP instead of TLS to the endpoint." | ||
| required: false | ||
| default: 'false' | ||
| access-key: | ||
| description: "S3 access key (pass from the org secret HZ_CI_CACHE_S3_ACCESS_KEY)." | ||
| required: true | ||
| secret-key: | ||
| description: "S3 secret key (pass from the org secret HZ_CI_CACHE_S3_SECRET_KEY)." | ||
| required: true | ||
| use-fallback: | ||
| description: "Fall back to the GitHub-hosted cache if the S3 endpoint is unreachable." | ||
| required: false | ||
| default: 'true' | ||
|
|
||
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - name: Cache Rust Cargo modules in S3 (RGW) | ||
| uses: tespkg/actions-cache@e07e2d4953dc8c020d447363e5064e36d04f3cf9 # v1.10.2 | ||
| with: | ||
| endpoint: ${{ inputs.endpoint }} | ||
| region: ${{ inputs.region }} | ||
| bucket: ${{ inputs.bucket }} | ||
| insecure: ${{ inputs.insecure }} | ||
| accessKey: ${{ inputs.access-key }} | ||
| secretKey: ${{ inputs.secret-key }} | ||
| use-fallback: ${{ inputs.use-fallback }} | ||
| retry: 'true' | ||
| retry-count: '3' | ||
| path: | | ||
| ${{ inputs.cargo-home }}/bin/ | ||
| ${{ inputs.cargo-home }}/registry/index/ | ||
| ${{ inputs.cargo-home }}/registry/cache/ | ||
| ${{ inputs.cargo-home }}/git/db/ | ||
| target/ | ||
| key: ${{ runner.os }}-${{ runner.arch }}-cache-rust-${{ inputs.cache-version }}-${{ hashFiles(inputs.cache-hashfiles-search-path) }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-${{ runner.arch }}-cache-rust-${{ inputs.cache-version }}- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
retry-countmay be silently ignoredretryis a documented input fortespkg/actions-cache, butretry-countdoes not appear in the published action interface. GitHub Actions silently discards unrecognisedwith:keys, so all three actions would retry with the action's default count (likely 1–3) rather than the intended value of3. If you want a deterministic retry count, verify this input is supported at the pinned commit; otherwise remove it to avoid misleading configuration. The same pattern appears inrust/cache-s3andnode/cache-pnpm-s3.Prompt To Fix With AI