Run the LTS test suite on ALCF GitLab CI, bridged to GitHub PRs - #613
Open
michel2323 wants to merge 4 commits into
Open
Run the LTS test suite on ALCF GitLab CI, bridged to GitHub PRs#613michel2323 wants to merge 4 commits into
michel2323 wants to merge 4 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #613 +/- ##
==========================================
+ Coverage 78.51% 79.97% +1.45%
==========================================
Files 50 50
Lines 3496 3496
==========================================
+ Hits 2745 2796 +51
+ Misses 751 700 -51 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
.gitlab-ci.yml: two-stage pipeline on the ALCF GitLab mirror (gitlab-ci.alcf.anl.gov/mschanen/oneAPI-jl), which pull-mirrors this repository. The Aurora PBS debug queue caps 1-node jobs at 1 h, so the support-library build and env instantiation/precompilation run on the login-node shell runner into a persistent depot on flare, and only the test run goes through PBS. Manifests are gitignored, so the build stage devs the checkout into the test env (a plain instantiate would silently test the registry release) and ships test/Manifest.toml to the batch job as an artifact. The runtime JIT gets the single-target "native,-avx512fp16" via -C (multi-target strings are image-only syntax), working around the same host-Float16 miscompile ci.yml documents. .github/workflows/alcf.yml: surfaces the GitLab pipeline as a PR check. Push to main and same-repo PRs force a pull-mirror sync and poll the pipeline for the head SHA; fork PRs run only when a maintainer applies the alcf-ci label (refs-only push as gh-pr-N — the workflow never checks out or executes PR code; the Aurora-side execution of labeled fork code is the deliberate, opt-in exception). Validated end-to-end on the mirror: build 96 s warm, full LTS suite green in ~19 min wall including queue wait. The existing self-hosted LTS job stays during burn-in and can be retired separately.
…rom GitLab The ALCF GitLab pipeline replaces the self-hosted LTS job, so ci.yml goes away (buildkite keeps covering the rolling stack). The docs build never executes GPU code — only `modules = [oneAPI]` for docstrings — so it moves to ubuntu-latest with versioninfo() guarded; this keeps Documenter's native deploy (DOCUMENTER_KEY, PR previews) instead of recreating it on GitLab. Coverage parity moves into the GitLab pipeline: the test job runs with --code-coverage=user and collapses .cov files to lcov.info (tooling staged into the shared depot by the build job, since compute nodes should not need the network), and a token-gated, allow_failure upload job on the shell runner submits to Codecov against the GitHub slug. Requires a CODECOV_TOKEN CI/CD variable on the GitLab project.
Member
Author
|
Only the default Julia version loaded via |
ParallelTestRunner's default uses the full node (workers spread over the 12 tiles via ONEAPI_TEST_SPREAD_GPUS); the explicit --jobs=4 was inherited from pbs_test.sh's conservative default.
The ALCF GitLab instance requires a login, so the bridge fetches every job trace after the pipeline finishes and prints them as collapsible groups in the Actions log, readable by any contributor.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds Aurora CI through ALCF's GitLab CI service while keeping GitHub as the source of truth, and retires the GitHub Actions self-hosted runner it replaces.
How it works
.gitlab-ci.yml(ignored by GitHub CI): a three-stage Aurora pipeline. The PBSdebugqueue caps 1-node jobs at 1 h, so the support-library build and env instantiation/precompilation run on the login-node shell runner into a persistent depot on flare, and only the test run itself goes through PBS. The build stage devs the checkout into the test env (manifests are gitignored, and a plain instantiate would silently test the registry release) and shipstest/Manifest.tomlto the batch job as an artifact. The test job runs with--code-coverage=user, collapses coverage tolcov.info, and a token-gatedallow_failurejob uploads to Codecov against this repo's slug — the LTS-gated branches are unreachable on the rolling stack, so this is their only coverage signal..github/workflows/alcf.yml: makes the GitLab pipeline visible as this PR check. Pushes to main and same-repo PRs force a pull-mirror sync and poll the pipeline for the head SHA; fork PRs run only when a maintainer applies thealcf-cilabel (the workflow moves refs only and never executes PR code — the Aurora-side execution of labeled fork code is the deliberate, opt-in exception, since GitLab CI jobs run as an ALCF user).ci.ymlremoved: the self-hosted LTS job is superseded by the above; buildkite keeps covering the rolling stack.modules = [oneAPI]for docstrings and has no doctests, so it moves from the self-hosted runner toubuntu-latestwithversioninfo()guarded. This keeps Documenter's native deploy (DOCUMENTER_KEY, PR previews) instead of recreating deployment from GitLab.Validation
Iterated to green on a scratch branch of the mirror: build stage 96 s warm, full LTS suite green in ~19 min wall including PBS queue wait (pipeline 58632). The "LTS stack (ALCF GitLab CI, Aurora)" check on this PR is the same-repo bridge path exercising itself end to end; the docs check now validates the GPU-less docs build.
Uses the
ALCF_GITLAB_TOKENrepo secret (GitLab project access token,api+write_repository). Codecov upload activates once aCODECOV_TOKENCI/CD variable is set on the GitLab project.🤖 Generated with Claude Code