CI: add timeout and retry to apt-get install steps - #624
Conversation
Bare `sudo apt-get install -y` with no `timeout-minutes` and no retry let a stalled Ubuntu mirror hang CI for hours — two 358-minute hangs on 2026-08-18 across two edition repos ten minutes apart, and four more hangs on 2026-08-19 each cleared only by cancel-and-rerun. Every apt-installing step now carries a step-level `timeout-minutes` and runs up to three attempts with a per-attempt `timeout`, so a stalled fetch fails fast and a transient mirror outage is ridden out rather than burning the job. Package lists are unchanged. Tracked in QuantEcon/project-translation#43.
There was a problem hiding this comment.
Pull request overview
This PR hardens GitHub Actions workflows against Ubuntu mirror stalls by adding step-level timeouts and retry logic around the LaTeX dependency installation steps, reducing the chance of multi-hour CI hangs in this Jupyter Book-based lecture build/publish pipeline.
Changes:
- Add
timeout-minutesto the “Install latex dependencies” steps. - Wrap
apt-get update+apt-get installin a 3-attempt retry loop with per-attempt timeouts and a short backoff.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| .github/workflows/ci.yml | Adds step timeout and retry/timeout loop around LaTeX apt installs in PR CI. |
| .github/workflows/publish.yml | Adds step timeout and retry/timeout loop around LaTeX apt installs in publish workflow. |
Suppressed comments (2)
.github/workflows/ci.yml:46
- The retry loop uses a single
timeout 9mcovering bothapt-get updateandapt-get install, but the PR description says the per-attempt timeouts are split (3m / 9m). As written, a slow update consumes install budget and also doesn’t match the stated behavior; splitting the timeouts makes the step’s intent clearer and more predictable.
for attempt in 1 2 3; do
timeout 9m sudo bash -c "apt-get -qq update && apt-get install -y $pkgs" && break
if [ "$attempt" = 3 ]; then echo "apt-get failed after 3 attempts"; exit 1; fi
.github/workflows/publish.yml:62
- The retry loop uses a single
timeout 9mcovering bothapt-get updateandapt-get install, but the PR description says the per-attempt timeouts are split (3m / 9m). As written, a slow update consumes install budget and also doesn’t match the stated behavior; splitting the timeouts makes the step’s intent clearer and more predictable.
for attempt in 1 2 3; do
timeout 9m sudo bash -c "apt-get -qq update && apt-get install -y $pkgs" && break
if [ "$attempt" = 3 ]; then echo "apt-get failed after 3 attempts"; exit 1; fi
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| - name: Install latex dependencies | ||
| timeout-minutes: 30 | ||
| run: | |
| - name: Install latex dependencies | ||
| timeout-minutes: 30 | ||
| run: | |
…er-attempt timeouts (4m/14m) Addresses Copilot review on QuantEcon/lecture-python.zh-cn#263: the cache.yml graphviz step was missed by a name match, and three 3m/9m attempts could never let a slow-but-progressing install finish. Now two attempts at 4m (graphviz) / 14m (texlive) inside the unchanged 10/30-minute step budgets.
…-get Addresses Copilot review on QuantEcon/lecture-intro.zh-cn#301: with timeout wrapping sudo, a timed-out attempt terminated sudo and the bash -c shell but could orphan apt-get holding the dpkg lock, making the retry block instead of retry. timeout is now the direct parent of each apt-get (inside sudo, -k 30s escalation): update 1m/2m, install 3m/12m for graphviz/texlive. Step budgets unchanged.
|
Closing unmerged (@mmcky, 2026-08-21): the apt hangs are accepted as a known flake for now rather than investing further here — the QuantEcon/actions container templates remove the apt step entirely, and that deployment is the medium-term fix. Branch left in place for reference; tracked in QuantEcon/project-translation#43. |
Bare
sudo apt-get install -ywith notimeout-minutesand no retry lets a stalled Ubuntu mirror hang CI for hours: two 358-minute hangs on 2026-08-18 across two edition repos ten minutes apart (a mirror outage), four more hangs on 2026-08-19 (Install latex dependencies77/15/16 min,Graphics Support21 min) each cleared only by cancel-and-rerun, and a 59-minute slow-but-green install on 2026-08-20.This PR hardens every apt-installing step in the workflows that carry one (
ci.yml,cache.yml,publish.ymlwhere present):timeout-minutes(10 for graphviz, 30 for the texlive set), so a hang can no longer take the six-hour job limit;timeout(3m / 9m) and a 30-second pause, so a transient mirror outage is ridden out rather than failing the build — a timeout alone would only turn a six-hour hang into a fast red build.Package lists are unchanged. The apt steps are edition-local (the shared
QuantEcon/actions/templatescarry none), so this lands per repo. Part of the five-repo sweep tracked in QuantEcon/project-translation#43.