Skip to content

ci: optional apt repository per build (aptly repo.sh, matching apt.new) - #35

Open
igorpecovnik wants to merge 2 commits into
mainfrom
ci-reprepro-repo-option
Open

ci: optional apt repository per build (aptly repo.sh, matching apt.new)#35
igorpecovnik wants to merge 2 commits into
mainfrom
ci-reprepro-repo-option

Conversation

@igorpecovnik

@igorpecovnik igorpecovnik commented Jul 27, 2026

Copy link
Copy Markdown
Member

Adds an opt-in path that builds a complete, signed apt repository with the same aptly tooling and keys production usestools/repository/repo.sh, the version armbian.github.io runs to build https://repo.armbian.com — so the layout matches apt.new (dists by release codename, pool by component). Groundwork for per-pull-request repositories.

Supersedes the earlier reprepro approach in this PR — switched to aptly so per-PR repos are byte-structurally identical to the production repo.

What it does

Folded into publish-debs-to-repo, reusing its single debs-to-repo-download. After the debs are downloaded (and before the rsync that moves them away):

  1. checks out the build repo's tools at repo_tool_ref (the production repo.sh);
  2. runs tools/repository/repo.sh -c update -k -i <output/debs[-beta]> -o output/repository → publishes dists/<release> + pool/<component> under output/repository/public, signing the top-level Release files with the keys imported above (GPG_KEY3 = Armbian repo, GPG_KEY4 = Igor);
  3. rsyncs the contents of public/ to storage/repositories/<path> → served at https://repo.armbian.com/repositories/<path>.

Both steps are continue-on-error, so the optional repo can never block the raw-deb publish.

Depends on armbian/build#10304

That PR brings the production repo.sh (+ 6 review fixes) onto main. Until it lands, repo_tool_ref temporarily points at its branch singlemode-to-main so this is testable now. Once #10304 merges, set repo_tool_ref to main (then it just uses the normal build checkout).

Toggle

  • build-all.yml: manual generateRepository input (default no).
  • Reusable inputs: generaterepository_default (default no), repository_target_path (→ target_path), repo_tool_ref (→ singlemode-to-main).

Notes

  • Only complete-artifact-matrix.yml + build-all.yml change; actionlint-clean on the touched steps.
  • armbian/build#10301 / #10303 remain valid fixes for the framework's debs-to-repo-reprepro, but are no longer on this path.

@igorpecovnik

Copy link
Copy Markdown
Member Author

First live test (run 30383424520, generateReprepro=yes on this branch) surfaced several issues — now addressed:

  1. reprepro built nothing — the generated reprepro.sh aborted on the first missing deb (Error 2 opening ...debset -e before export). Partial builds always miss some debs, so this is fatal in practice. → Fixed properly in repo-reprepro: skip debs missing on disk instead of aborting the whole repo build#10301 (skip debs not present on disk). This PR depends on it.
  2. Signing failedGPG_KEY3/GPG_KEY4 were not in armbian/ci secrets (only in armbian.github.io). @igorpecovnik has since added them to armbian/ci. Also hardened here: the two keyids are trimmed so a missing import yields "" (unsigned) instead of a stray space.
  3. Upload would failstorage/repositories/<path> did not exist and rsync will not create missing parents. Now the step pre-creates the tree over ssh and skips (with a warning) when the build produced no dists/.
  4. Many unauthorized ORAS fetches — that was a fresh-trunk partial state (artifacts not yet published), not a bug; ci: create + read releases in this repo (self-contained test CI) #1 makes the repo tolerant of it.

Re-test once armbian/build#10301 is merged (the reprepro job checks out build main). Still open: confirm the served storage/repositories/ path.

@igorpecovnik

Copy link
Copy Markdown
Member Author

Second test run (30400943657) — the important parts now work: GPG import succeeds (both keys imported, SignWith: 1EB30FAF236099FE 93D6889F9F0E78D5), and #10301 prevents the abort (skips missing debs instead of dying).

But it produced a ~214 MB log / 7-min step. Root cause: the reprepro build ran under bash -x, which reprints the (264 KB) for one_deb in <2165 debs> header on every iteration — ~half a GB of trace for one target. Compounded by #10301 echoing one line per missing deb.

Fixes:

Note: the log bloat obscured the tail, so I could not confirm export/signing/upload completed in that run — a clean re-test after both land will show it. Still open: the served storage/repositories/ path.

…oduction)

Adds an opt-in path that builds a complete, signed apt repository with the same
aptly tooling and keys production uses (tools/repository/repo.sh, the version
armbian.github.io runs to build repo.armbian.com), so the layout matches
apt.new (dists by release codename, pool by component) -- groundwork for
per-pull-request repositories.

Folded into the existing publish-debs-to-repo job, reusing its single
debs-to-repo-download: after the debs are downloaded (and before the rsync that
moves them away with --remove-source-files), an opt-in step checks out the build
repo's tools at repo_tool_ref and runs:

    tools/repository/repo.sh -c update -k -i <output/debs[-beta]> -o output/repository

which publishes dists/<release> + pool/<component> under output/repository/public
and signs the top-level Release files with the keys imported above (GPG_KEY3 =
Armbian repo key, GPG_KEY4 = Igor). A companion step rsyncs the contents of
public/ to storage/repositories/<path> (served at
https://repo.armbian.com/repositories/<path>). Both steps are continue-on-error
so the optional repo can never block the raw-deb publish.

repo_tool_ref TEMPORARILY points at the armbian/build PR branch
(singlemode-to-main) that carries the production repo.sh; set it to "main" once
armbian/build#10304 lands.

Wiring:
- complete-artifact-matrix.yml: inputs renamed to generaterepository_default /
  repository_target_path, new repo_tool_ref input; reprepro steps replaced with
  the repo-tool checkout + repo.sh build + public/ upload.
- build-all.yml: manual input renamed generateReprepro -> generateRepository.

Replaces the earlier reprepro approach (debs-to-repo-reprepro); armbian/build
#10301 / #10303 remain valid fixes for that framework command but are no longer
on this path.

Signed-off-by: Igor Pecovnik <igor@armbian.com>
@igorpecovnik
igorpecovnik force-pushed the ci-reprepro-repo-option branch from 67b0e52 to 840ff87 Compare July 29, 2026 11:37
@igorpecovnik igorpecovnik changed the title ci: optional reprepro apt repository (build & upload) ci: optional apt repository per build (aptly repo.sh, matching apt.new) Jul 29, 2026
The "Build signed apt repository (aptly repo.sh)" step produced an empty
repository: every aptly publish reported "publishing from empty source"
and zero packages, so pool/ shipped empty.

Root cause is a BETA mismatch between steps. The version, build and
nightlybuild-passthrough steps resolve nightly/stable as
`github.event.inputs.nightlybuild || inputs.nightlybuild_default`, so a
run driven by the parent workflow's nightlybuild_default=yes (dispatch
input unset) builds nightly -- trunk REVISION (26.8.0-trunk.5), debs
pushed to the beta namespace. But the debs-to-repo download and repo.sh
steps keyed off `github.event.inputs.nightlybuild == 'true'` alone, which
ignores nightlybuild_default and evaluates to BETA=no. The download then
looked in the stable namespace, logged "Missing deb" for every package,
left output/debs empty, and repo.sh had nothing to import.

Resolve BETA in both steps the same way the build does -- nightly unless
explicitly 'no' (mirrors the Determine version step's `[ "$NIGHTLY" = "no" ]`)
-- normalized to yes/no. Download and repo.sh now target the same
namespace/dir the build wrote to, so the signed repo is populated.

Signed-off-by: Igor Pecovnik <igor@armbian.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant