Skip to content

Tag what a release uploaded instead of what it was meant to upload #568

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
andiwand wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Tag what a release uploaded instead of what it was meant to upload #568

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 17 additions & 29 deletions .github/scripts/resolve-version.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,18 @@
# Works out which version a release run is building, and refuses the runs that
# cannot sensibly build one.
#
# There is no version number in the repository: it is the git tag, which
# app/build.gradle turns into a version name and a version code (v4.8.0 -> 4.8.0
# and 40800, two digits per part). What is left to decide is which string gradle
# is handed, and that is only interesting when the run has no tag to read:
# There is no version number in the repository: it comes in as the release run's
# `version` input, and app/build.gradle turns it into a version name and a version
# code (v4.8.0 -> 4.8.0 and 40800, two digits per part).
#
# tag push the tag, and the version input has to agree with
# it or stay empty - the apk of a run is attached to
# the release of the tag it ran on, so building
# anything else would file it there under the wrong
# version
# dispatched off a branch the version input, which is how a release whose
# upload half failed gets finished off the branch it
# was cut from
# neither only a dry run, on gradle's unversioned fallback.
# uploading that would mean uploading a version code
# the store refuses, six minutes into the run
# a version input that version
# no input only a dry run, on gradle's unversioned fallback. uploading
# that would mean uploading a version code the store refuses,
# six minutes into the run
#
# It is an input rather than the tag the run was pushed on because a tag written
# before the upload names a commit that may never ship; release.yml writes the
# tags afterwards instead.
#
# The shape is checked here rather than left to gradle, which checks it again and
# is the one that counts: a typo in a dispatched version should not cost the
Expand Down Expand Up @@ -50,22 +46,15 @@ def fail(message):
return 1


def resolve(tag, given, uploads, log=print):
def resolve(given, uploads, log=print):
"""The version to build, or "" for none. Raises ValueError with the reason."""
tag, given = tag.strip(), given.strip()

if tag and given and given.removeprefix("v") != tag.removeprefix("v"):
raise ValueError(
f"the version input ({given}) is not the tag this ran on ({tag}). "
"leave it blank to build the tag."
)
version = given.strip()

version = tag or given
if not version:
if uploads != "none":
raise ValueError(
"nothing to take a version from. push this as a v* tag, dispatch "
"it on one, or fill in the version input."
"nothing to take a version from: fill in the version input, or "
"set uploads to none to build without publishing."
)
log("no version given - building gradle's unversioned fallback")
return ""
Expand All @@ -83,8 +72,7 @@ def main(argv=None):
parser = argparse.ArgumentParser(
description="Resolve the version a release run builds."
)
parser.add_argument("--tag", default="", help="tag the run was triggered by, if any")
parser.add_argument("--input", default="", help="version input of a dispatched run")
parser.add_argument("--input", default="", help="version input of the run")
parser.add_argument(
"--uploads",
default="none",
Expand All @@ -93,7 +81,7 @@ def main(argv=None):
args = parser.parse_args(argv)

try:
version = resolve(args.tag, args.input, args.uploads)
version = resolve(args.input, args.uploads)
except ValueError as reason:
return fail(str(reason))

Expand Down
106 changes: 106 additions & 0 deletions .github/workflows/attach-apk.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
name: attach apk

# Puts the signed pro apk on the github release of a v* tag - the sideloadable
# copy every release up to v4.6 carried.
#
# A workflow of its own, because the two things happen days apart: release.yml
# uploads to play's internal track, and the v* tag is written afterwards, once the
# release is actually live. See the "Tags" section in the README.
#
# It attaches the apk that was uploaded rather than building a second one: the
# release run archives both apks, and this fetches that artifact.

on:
push:
tags:
- 'v*'

permissions:
contents: write # the release asset
actions: read # the release run's artifact

jobs:
attach:
runs-on: ubuntu-24.04
steps:
# the build tags are what says which commit went out, so all of them
- name: checkout
uses: actions/checkout@v7
with:
fetch-depth: 0

# a v* tag means "this shipped", so there has to be an upload it names, and it
# has to be the commit that was uploaded. this is the only thing that catches
# the two drifting apart
- name: find the upload this tag names
id: upload
env:
tag: ${{ github.ref_name }}
run: |
build_tag="build/pro/v${tag#v}"
if ! git rev-parse -q --verify "refs/tags/$build_tag" > /dev/null; then
echo "::error::no $build_tag, so $tag names no pro upload. dispatch release.yml first."
exit 1
fi

built=$(git rev-list -n1 "$build_tag")
if [ "$built" != "$GITHUB_SHA" ]; then
echo "::error::$tag is at $GITHUB_SHA but pro was built from $built. move the tag: git tag -f $tag $build_tag^{}"
exit 1
fi

# both flavors always go out together, so a missing lite means the release
# is half done. a lite built elsewhere is legitimate - that is what
# finishing a half uploaded release looks like - but worth saying out loud
lite_tag="build/lite/v${tag#v}"
if ! git rev-parse -q --verify "refs/tags/$lite_tag" > /dev/null; then
echo "::error::no $lite_tag, so lite never went out under $tag"
exit 1
fi
lite=$(git rev-list -n1 "$lite_tag")
if [ "$lite" != "$GITHUB_SHA" ]; then
echo "::warning::lite $tag was built from $lite, not $GITHUB_SHA"
fi

run=$(git for-each-ref --format='%(contents)' "refs/tags/$build_tag" | sed -n 's/^run: //p')
if [ -z "$run" ]; then
echo "::error::$build_tag names no run, so the apk it uploaded cannot be found"
exit 1
fi
echo "run=$run" >> "$GITHUB_OUTPUT"

# artifacts are kept 90 days; a version tag written long after the upload finds
# nothing, and download-artifact says so far less clearly
- name: check the apks are still there
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
run_id: ${{ steps.upload.outputs.run }}
run: |
expired=$(gh api "repos/$GH_REPO/actions/runs/$run_id/artifacts" \
--jq '.artifacts[] | select(.name == "apks") | .expired')
if [ "$expired" != "false" ]; then
echo "::error::run $run_id no longer has its apks, so there is nothing to attach"
exit 1
fi

- name: fetch the apks
uses: actions/download-artifact@v8
with:
name: apks
run-id: ${{ steps.upload.outputs.run }}
github-token: ${{ github.token }}

# pro alone, the way the release page has always had it. the release itself
# stays a human decision - this only fills in its apk
- name: attach the pro apk to the github release
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
tag: ${{ github.ref_name }}
run: |
if ! gh release view "$tag" > /dev/null 2>&1; then
echo "::error::$tag has no github release to attach the apk to. create it, then re-run this job."
exit 1
fi
gh release upload "$tag" pro/release/app-pro-release.apk --clobber
132 changes: 62 additions & 70 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,22 @@ name: release
# internal track, and only ever that one. a wider track is a promotion in
# the play console, which moves the very bundle that was tested rather than
# uploading a second one, and is where the release notes are written anyway
# 3. on a tag, attach the signed pro apk to that tag's github release, which
# every release up to v4.6 carried and the laptop build produced by hand.
# the release has to exist already. this is a second job, so that it can
# be re-run without re-running the upload in step 2
# 3. tag the commit each upload was built from, as build/<flavor>/<version>
#
# Both flavors go out together, the way they always have: they share a version
# and release notes, and a release of one without the other is not a thing that
# has ever been wanted.
#
# The version is the tag and nothing else. The repository holds no version number
# at all any more - gradle takes it from the -Podr.version passed below, and turns
# v4.8.0 into version code 40800, two digits per part. A dispatched run has no tag
# to read, so it fills in the version input instead.
# Nothing here is triggered by a tag. A tag pushed before the build is a promise
# the run can fail to keep - it can die before the upload, or get only half of it
# through, and what reaches the store is then built from some other commit. So the
# version comes in as an input and the tags are written afterwards, naming what
# really went up. The v* tag stays a human decision, taken once the release is
# live; attach-apk.yml is what runs on it.
#
# The version is the input and nothing else. The repository holds no version
# number at all - gradle takes it from the -Podr.version passed below, and turns
# v4.8.0 into version code 40800, two digits per part.
#
# Requires these repository secrets (see the "Release signing" section in the
# README for what they mean):
Expand Down Expand Up @@ -48,25 +51,20 @@ on:
type: choice
options: [both, pro, lite, none]
default: both
# a tag push needs nothing here: the tag is the version. this is for dispatched
# runs, which have no tag to read - finishing a half uploaded release off the
# branch it was cut from, or putting a real version on a dry run
version:
description: version to build, e.g. v4.8.0 - defaults to the tag
description: version to build, e.g. v4.8.0 - required unless uploads is none
type: string
push:
tags:
- 'v*'

concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false

permissions:
contents: read
# to tag the commit an upload was built from
contents: write

env:
uploads: ${{ inputs.uploads || 'both' }}
uploads: ${{ inputs.uploads }}

jobs:
release:
Expand All @@ -92,19 +90,32 @@ jobs:
- name: checkout
uses: actions/checkout@v7

# the version is not in the checkout - it is the tag, handed to gradle below as
# -Podr.version. The script says which runs can build which version, and why;
# it is here, right behind the checkout it needs, because a run that cannot name
# a version should end before the six minutes of setup and building, not after
# the version is not in the checkout - it comes in as an input, and reaches gradle
# below as -Podr.version. Here, right behind the checkout it needs, so that a run
# that cannot name a version ends before six minutes of setup and building
- name: resolve version
id: version
# through the environment rather than interpolated into the command: a tag name
# and a dispatch input are both strings from outside the workflow, and a run:
# line is the one place where that would be a shell injection
# through the environment rather than interpolated into the command: the input
# is a string from outside the workflow, and a run: line is the one place where
# that would be a shell injection
env:
tag: ${{ github.ref_type == 'tag' && github.ref_name || '' }}
given: ${{ inputs.version }}
run: .github/scripts/resolve-version.py --tag "$tag" --input "$given" --uploads "$uploads"
run: .github/scripts/resolve-version.py --input "$given" --uploads "$uploads"

# same reason: play refuses a version code it has already accepted, so a version
# that is already tagged as uploaded would only fail once the build was done
- name: check the version has not gone out
if: ${{ env.uploads != 'none' }}
env:
version: ${{ steps.version.outputs.version }}
run: |
for flavor in $([ "$uploads" = both ] && echo pro lite || echo "$uploads"); do
tag="build/$flavor/v${version#v}"
if git ls-remote --exit-code --tags origin "$tag" > /dev/null 2>&1; then
echo "::error::$tag exists, so $flavor $version has already been uploaded"
exit 1
fi
done

- name: install ninja
run: sudo apt-get install -y ninja-build
Expand Down Expand Up @@ -194,9 +205,10 @@ jobs:
path: app/build/outputs/bundle/*/*.aab
if-no-files-found: error

# both, and not only on the release: a dispatched run has no tag to attach
# anything to, and this is how a release gets test flown - including lite,
# which is otherwise only installable once it is live in the store
# how a release gets test flown - including lite, which is otherwise only
# installable once it is live in the store. attach-apk.yml fetches the pro one
# from here when the version tag is pushed, so this is also what ends up on the
# release page rather than a second build of the same commit
- name: Artifact apks
uses: actions/upload-artifact@v7
with:
Expand All @@ -205,57 +217,37 @@ jobs:
if-no-files-found: error
compression-level: 0

# each flavor is tagged as it lands rather than both at the end: the release
# that goes up half is the one the record exists for, and a lite failure must
# not take pro's tag with it. per flavor for the same reason - the two halves
# can be finished from different commits
- name: upload to play store
if: ${{ env.uploads != 'none' }}
env:
ODR_PLAY_JSON_KEY: ${{ runner.temp }}/fastlane_google_play.json
version: ${{ steps.version.outputs.version }}
run: |
case "${{ env.uploads }}" in
pro) lanes="uploadPro" ;;
lite) lanes="uploadLite" ;;
*) lanes="uploadPro uploadLite" ;;
esac
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"

# no track: the Fastfile's DEFAULT_TRACK is internal, and this workflow
# has no way of naming another one. everything wider is a promotion in
# the play console
for lane in $lanes; do
bundle exec fastlane android "$lane"
for flavor in $([ "$uploads" = both ] && echo pro lite || echo "$uploads"); do
case "$flavor" in
pro) bundle exec fastlane android uploadPro ;;
lite) bundle exec fastlane android uploadLite ;;
esac

tag="build/$flavor/v${version#v}"
# attach-apk.yml reads the run back out of the message
git tag -a "$tag" \
-m "$flavor v${version#v} uploaded to the play store internal track" \
-m "run: $GITHUB_RUN_ID"
git push origin "$tag"
echo "tagged \`$GITHUB_SHA\` as \`$tag\`" >> "$GITHUB_STEP_SUMMARY"
done

- name: drop credentials
if: always()
run: rm -f "${RUNNER_TEMP}/fastlane_google_play.json" "${RUNNER_TEMP}/google_play.keystore"

# needs: release, so the release page never offers an apk for a version that
# never reached play - and a job of its own, so that a failure here can be
# re-run on its own. re-running the release job is not an option once
# fastlane has been through it: play rejects a second upload of a version
# code it has already seen, so the retry would die before ever getting here
attach:
needs: release
# inputs, not env: a job level if cannot see the env context, and an unset
# input on a tag push is not 'none' either way
if: ${{ github.ref_type == 'tag' && inputs.uploads != 'none' }}
runs-on: ubuntu-24.04
permissions:
contents: write
steps:
- name: fetch the apks
uses: actions/download-artifact@v8
with:
name: apks

# pro alone, the way the release page has always had it. the release itself
# stays a human decision - this only fills in its apk, and says so rather
# than inventing one
- name: attach the pro apk to the github release
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
tag: ${{ github.ref_name }}
run: |
if ! gh release view "$tag" > /dev/null 2>&1; then
echo "::error::$tag has no github release to attach the apk to. create it, then re-run this job."
exit 1
fi
gh release upload "$tag" pro/release/app-pro-release.apk --clobber
Loading
Loading