Skip to content

internal: stop re-polling QUEUED checks during the Dependabot batch - #6141

Open
prql-bot wants to merge 2 commits into
mainfrom
daily/review-runs-30893175261
Open

internal: stop re-polling QUEUED checks during the Dependabot batch#6141
prql-bot wants to merge 2 commits into
mainfrom
daily/review-runs-30893175261

Conversation

@prql-bot

@prql-bot prql-bot commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Daily review of the previous night's tend runs surfaced one finding that clears both gates: tend-review sessions on the daily Dependabot batch spend 20–45 minutes re-entering the CI poll loop against checks that structurally cannot settle inside the session.

What happens

Five Dependabot PRs (#6130, #6132#6135) plus pre-commit-ci's #6131 opened over 2m13s — 17:14:32 to 17:16:45 — on 2026-08-03. On #6130's branch alone, five tests runs were created at 17:14:37; the cancel-in-progress concurrency group culled four, and the survivor — run 30835855220 — took 73 minutes end to end (17:14:3718:27:46), most of it QUEUED behind the five sibling matrices.

The poll loop under CI Monitoring in the bundled running-in-ci skill caps at 9 minutes per invocation, so each review re-entered it repeatedly and still ended unverified:

Review Job duration Poll rounds Outcome
#6130 (run) 41 min 4 replacement run still queued at exit
#6131 (run) 21 min 2 two checks still QUEUED at exit
#6133 (run) 40 min 4 "the Rust matrix hadn't finished"
#6135 (run) 41 min 4 did settle
#6134 (run) 45 min 3 did settle (after a legitimate rerun of a flaked Windows job)

The reviews themselves were good — each one diagnosed the stale check-ok-to-merge red as a cancellation-cascade artifact rather than treating it as real, and each named the checks it had not observed reach terminal rather than implying coverage. The waste is purely the extra rounds. In #6130's own words: "Polling has run ~29 minutes; the replacement tests run is still queued (concurrency-queued) … It won't settle in this session." — three rounds after that observation, it still hadn't.

The rule

The two runs that did settle (#6134, #6135) were watching checks that were progressing; the three that wasted rounds were watching checks stuck in QUEUED. That distinction is the discriminator, so the rule keys on it rather than banning re-entry outright: stop after one round when pending checks are still QUEUED, keep going while they are IN_PROGRESS.

Gate assessment

  • Evidence level: High — structural. Runner saturation during the daily batch is deterministic, not a model choice. 3 clear occurrences this window, plus 3 historical entries in the tracking issue (chore: bump minijinja from 2.20.0 to 2.21.0 #6063 at 24 min on 2026-07-07, chore: bump tokio from 1.52.3 to 1.53.1 #6111 at 42 min on 2026-07-28, chore: bump github/codeql-action from 4 to 4.37.3 #6120 at 36 min on 2026-07-31). High needs 2–3; this is 6.
  • Change type: new short section in an existing repo overlay — the "new paragraph" bar of 3+ occurrences is met.
  • Not a bundled defect: the generic guidance is already correct (poll once, then report unverified). What the model lacks is the repo-specific fact that a QUEUED PRQL matrix during the batch means 40–75 minutes, which only an overlay can supply.
Session evidence

Runs analyzed: 30835853083, 30835879825, 30835900705, 30835922625, 30836026660, 30838560578, 30846380951, 30852089164, 30858704413, 30886931468, 30887510480, 30887553479, 30887961990, 30887996356, 30888281468, 30888469905, 30888779216, 30888815275.

Poll-round counts come from counting seq 1 9 loop invocations in each session's JSONL. The five concurrent tests runs on #6130's branch: 30835854409, 30835854490, 30835854594, 30835854618 (all cancelled) and 30835855220 (success).

@prql-bot prql-bot added the review-runs Findings from tend-review-runs daily analysis label Aug 4, 2026

@prql-bot prql-bot left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two notes on the new section — both about making the rule unambiguous for a future run that has only this text to go on.

One gap without an obvious fix: applying the rule requires seeing which states are pending, but the pending() helper in CI Monitoring collapses the states inside jq and returns only a count, so a run following that recipe verbatim never observes QUEUED vs IN_PROGRESS. Naming the query in this section (gh pr view <n> --json statusCheckRollup and reading .status // .state) would make the rule mechanically applicable rather than dependent on the model deciding to look.

(Skipped /code-review here: .claude/ is bind-mounted read-only in the sandbox, so the working tree shows the base version of the file and the pass would have seen an empty diff. Reviewed against git show HEAD:.claude/skills/running-tend/SKILL.md instead.)

Comment thread .claude/skills/running-tend/SKILL.md Outdated
Comment thread .claude/skills/running-tend/SKILL.md Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-runs Findings from tend-review-runs daily analysis

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant