fix(sweep): replace the reply-reaction shortcut with the four-condition trigger gate - #2486
fix(sweep): replace the reply-reaction shortcut with the four-condition trigger gate#2486carlos-alm wants to merge 1 commit into
Conversation
…on trigger gate Step 2g decided "is Greptile satisfied?" by checking for a positive reaction on THE MOST RECENT NON-GREPTILE COMMENT - i.e. on one of your own replies. A 👍 on a reply is not a re-review of your code. Under that rule the re-trigger is skipped while the fix sits un-reviewed, which is how a sweep reports a PR as converged when Greptile has never seen the change. Transplant the four-condition gate the fleet standardised on. Satisfied now requires ALL of: an @greptileai TRIGGER comment exists and really mentions Greptile (a literal inside a code span, fenced block, indented block or HTML comment notifies nobody and is not a trigger); Greptile reacted positively TO THAT TRIGGER; Greptile posted nothing since; and Greptile has reviewed the CURRENT head, established from its own `Last reviewed commit` marker rather than a commit timestamp - Greptile re-reviews by EDITING its summary IN PLACE, so timestamps cannot observe a completed re-review. Every fetch failure fails safe by posting. Add Step 2i (final mandatory re-trigger) and renumber Return result to 2j, matching the fleet layout. The gate is meant to run twice and its own header names Step 2i; that header doubles as the sentinel the shared test suite locates it by, so aligning this repo's numbering keeps one identical suite proving every copy - the alternative, rewording the sentinel per repo, would fork the contract. The 50-trigger cap is preserved and now stated where it matters: the gate is not exempt from it, and Step 2g's prose says so. That prose previously taught the exact rule the gate replaces ("skip the actual trigger only if Greptile already reacted to your most recent reply"), contradicting the code beneath it. Verified with test_sweep_greptile_gate.py, ported alongside: it extracts the real fenced block from SKILL.md and runs it behind a stub gh under both bash and zsh. Three of its six cases are mutation tests that re-introduce the pre-fix behaviour and require the regression assertion to fail. 6/6. Wired via sweep-gate.yml, since a gate that lives in Markdown is otherwise never executed by anything. Refs: optave/data-retrieval-storage-svc#1021, optave/data-retrieval-storage-svc#930
Greptile SummaryThe PR replaces reply-reaction satisfaction with a four-condition Greptile trigger gate and adds a cross-shell fixture suite plus path-filtered CI. Two fetch-error paths still fail to uphold the gate’s stated fail-safe behavior.
Confidence Score: 3/5The PR should not merge until failed comment fetches consistently attempt the mandatory trigger rather than aborting or permitting timestamp-based satisfaction. The new gate can terminate without posting on its first fetch failure and can silently treat a later marker-fetch failure as an absent marker, allowing the documented inexact timestamp proxy to skip review of the current head. Files Needing Attention: .claude/skills/sweep/SKILL.md Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Run Step 2g or Step 2i gate] --> B[Fetch trigger comments]
B -->|Failure| X[Exit without posting]
B -->|Success| C[Check trigger reaction and later comments]
C --> D[Fetch current head]
D --> E[Fetch reviewed-commit markers]
E -->|Marker matches head| F[Skip trigger if all conditions hold]
E -->|No marker or fetch failure| G[Timestamp fallback]
G -->|Timestamp not newer| F
G -->|Timestamp newer or unavailable| H[Post @greptileai]
Reviews (1): Last reviewed commit: "fix(sweep): replace the reply-reaction s..." | Re-trigger Greptile |
| trigger_candidates=$(gh api repos/<repo>/issues/<number>/comments --paginate \ | ||
| --jq '.[] | select(.user.login != "greptile-apps[bot]") | "\(.id)\t\(.created_at)\t\((.body // "") | @json)"') \ | ||
| || { echo "FATAL: could not fetch trigger comments — aborting gate"; exit 1; } |
| reviewed_shas=$(gh api repos/<repo>/issues/<number>/comments --paginate \ | ||
| --jq '.[] | select(.user.login == "greptile-apps[bot]") | .body' 2>/dev/null \ | ||
| | grep -o 'Last reviewed commit:.*/commit/[0-9a-fA-F]\{40\}' \ | ||
| | grep -o '[0-9a-fA-F]\{40\}$' | tr 'A-Z' 'a-z') |
There was a problem hiding this comment.
Marker fetch failure permits false skip
When this issue-comments request fails after the initial fetch succeeded, the pipeline silently produces an empty reviewed_shas and enters the timestamp fallback. If the commit timestamp is no later than the trigger, the gate treats the head as reviewed and skips the mandatory trigger even though Greptile's current-head marker was never retrieved.
The bug
Step 2g decided "is Greptile satisfied?" by checking for a positive reaction on the most recent non-Greptile comment — that is, on one of your own replies. A 👍 on a reply is not a re-review of your code. Under that rule the re-trigger is skipped while the fix sits un-reviewed — which is how a sweep reports a PR as converged when Greptile has never seen the change.
The fix
Transplant the four-condition gate the fleet standardised on. Satisfied now requires all of:
@greptileaitrigger comment exists — and really mentions Greptile. A literal inside a code span, fenced block, indented block or HTML comment notifies nobody and is not a trigger.Last reviewed commitmarker — not a commit timestamp. Greptile re-reviews by editing its summary in place, so timestamps cannot observe a completed re-review; a timestamp proxy re-triggers commits Greptile has already approved. Timestamps survive only as the fallback when no marker parses.Every fetch failure fails safe by posting rather than assuming satisfaction.
Fitting it to this repo
Return resultrenumbered to 2j. The gate is designed to run twice, and its own header names Step 2i. That header doubles as the sentinel the shared test suite locates the gate by, so aligning this repo's numbering keeps one identical suite proving every fleet copy. I first tried the reverse — rewording the sentinel to say "Step 2h.1" to match the existing layout — and it immediately broke the suite's locator, which is the right outcome: that line is a fleet contract, not prose.Verification
test_sweep_greptile_gate.pyported alongside. It extracts the real fenced block fromSKILL.md— no copy — and runs it against fixture PR state behind a stubgh, under bothbashandzsh(zsh because it is the maintainers' interactive shell, and a/sweepguard in a sibling repo once silently passed under zsh while failing its job, since zsh does not word-split an unquoted parameter).Three of its six cases are mutation tests: they revert the gate to the pre-fix behaviour — remove the marker lookup, break the SHA regex, ignore a marker mismatch — and require the regression assertion to fail. Without them that assertion could pass with and against the bug it names.
6/6 on this branch. Wired into CI as
sweep-gate.yml, path-filtered to the skill + suite — without it nothing ever executes this gate: it lives in Markdown, so it is shipped, load-bearing and otherwise entirely unproven.Part of the fleet-wide sweep tracked by
optave/data-retrieval-storage-svc#1021.