ci: run fork-PR unit/lint/type checks offline from a prewarmed cache (#831)#845
Merged
Conversation
…831) Fork pull_request runs never receive a GitHub OIDC token, so .github/actions/setup-jfrog (which mints a JFrog PyPI token from that OIDC token) dies with "ACTIONS_ID_TOKEN_REQUEST_TOKEN: unbound variable" and every required check goes red on every fork PR — regardless of the diff (#831, reported against the trivial fork PR #671). Public PyPI is unreachable from the protected runners, so a fork run can obtain no package source at all. Adopt the Databricks SOP (mirrors databricks/dbt-databricks): a trusted warmer workflow that DOES get OIDC pre-builds the in-project .venv for every matrix leg and saves it to the Actions cache; fork PRs restore that .venv and run the tools directly from .venv/bin, fully offline, never touching JFrog. Everything still flows through JFrog (the warmer populates the cache from it); the fork just holds no credentials, so there is no pull_request_target / secret-exposure risk. - .github/workflows/warm-deps-cache.yml: new, sole writer of the forkvenv-* cache. Triggers on push:main (dep files), a daily schedule (beats GitHub's 7-day cache eviction), and workflow_dispatch with an optional pr_number that warms a fork's changed lockfiles (fetches ONLY poetry.lock/pyproject.toml, no source). Warms the full py x depset x extras matrix, priming mypy stubs too. - .github/actions/restore-deps: new composite action; restores the newest forkvenv-* entry for the leg via a restore-keys prefix and reports cache-hit. - .github/workflows/code-quality-checks.yml: each job tries restore-deps first and runs offline from .venv/bin on a hit; on a miss (all same-repo PRs, or a fork whose lockfiles changed pre-warm) it falls back to the original setup-poetry JFrog flow, unchanged. Same-repo PR behavior is unaffected. Scope: unit + lint + type checks (the fork-reachable required checks). e2e / kernel-e2e / integration / DBR-LTS stay merge-queue-gated (they need live warehouse secrets). Closes #831 Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
gopalldb
approved these changes
Jul 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fork PRs currently show all required checks red regardless of the diff (#831, reported against the trivial fork PR #671). Root cause: every job in
code-quality-checks.ymlinstalls deps viasetup-poetry→setup-jfrog, which mints a JFrog PyPI token from a GitHub OIDC token. GitHub deliberately withholds the OIDC token (ACTIONS_ID_TOKEN_REQUEST_TOKEN) from forkpull_requestruns, sosetup-jfrog'sset -uscript dies withunbound variablebefore any test runs. Public PyPI is unreachable from the protected runners, so a fork run can obtain no package source at all.This adopts the Databricks SOP for this exact problem (mirrors
databricks/dbt-databricks): invert the flow so a trusted workflow — which does get OIDC — pre-builds the environment and caches it, and the fork run replays it offline..github/workflows/warm-deps-cache.yml(new) — the sole writer of theforkvenv-*cache. Builds the in-project.venvfor the fullpython × depset × extrasmatrix from JFrog and saves each leg. Triggers:push:main(dep files), a daily schedule (beats GitHub's 7-day cache eviction), andworkflow_dispatchwith an optionalpr_numberthat warms a fork's changed lockfiles (fetches onlypoetry.lock/pyproject.toml— no source, no workflows). Primes mypy stub packages too..github/actions/restore-deps(new) — restores the newest warmed.venvfor the leg via arestore-keysprefix and reportscache-hit..github/workflows/code-quality-checks.yml— each job triesrestore-depsfirst and, on a hit, runs the tools directly from.venv/bin(notpoetry run, because even bootstrapping Poetry would hit JFrog). On a miss — every same-repo PR, or a fork whose lockfiles changed before a maintainer re-warmed — it falls back to the originalsetup-poetryJFrog flow, unchanged.Why this is safe
The fork run holds no credentials and needs none — it only reads a GitHub Actions cache and runs offline. Everything still flows through JFrog (the warmer populates the cache from it). No
pull_request_target, no secret exposure to fork code.Scope
Unit + lint + type checks (the fork-reachable required checks).
e2e/kernel-e2e/ integration / DBR-LTS stay merge-queue-gated — they need live warehouse secrets and are out of scope for #831.Fork dependency-change flow
If a fork PR changes
poetry.lock/pyproject.toml, its cache key won't match and CI misses the cache. A maintainer reviews the dep change, runs Actions → Warm Dependency Cache → Run workflow →pr_number = N, then the contributor re-runs CI.Test plan
actionlintclean on both new/changed workflows (only pre-existing custom-runner-label warnings remain, shared by all workflows).poetry runon the fork path is guarded bycache-hit != 'true';.venvis relocatable / interpreter restored viaactions/setup-pythonon the hit path.mainto seed the cache, then open a fork PR and confirm (a) the.venvrestores, (b)poetry installis skipped, (c)pytest/black/mypygenuinely run and report pass/fail, (d) no JFrog call on the fork job.Follow-ups for a maintainer
mainbefore the first fork PR relies on it.Closes #831
This pull request and its description were written by Isaac.