Skip to content

fix(sweep): replace the reply-reaction shortcut with the four-condition trigger gate - #2486

Open
carlos-alm wants to merge 1 commit into
mainfrom
fix/sweep-four-condition-gate
Open

fix(sweep): replace the reply-reaction shortcut with the four-condition trigger gate#2486
carlos-alm wants to merge 1 commit into
mainfrom
fix/sweep-four-condition-gate

Conversation

@carlos-alm

Copy link
Copy Markdown
Contributor

The bug

Step 2g decided "is Greptile satisfied?" by checking for a positive reaction on the most recent non-Greptile comment — that is, on one of your own replies. A 👍 on a reply is not a re-review of your code. Under that rule the re-trigger is skipped while the fix sits un-reviewed — which is how a sweep reports a PR as converged when Greptile has never seen the change.

The fix

Transplant the four-condition gate the fleet standardised on. Satisfied now requires all of:

  1. an @greptileai trigger comment exists — and really mentions Greptile. A literal inside a code span, fenced block, indented block or HTML comment notifies nobody and is not a trigger.
  2. Greptile reacted positively to that trigger (never to a reply).
  3. Greptile posted no issue-style or inline comment since.
  4. Greptile has reviewed the current head, established from its own Last reviewed commit marker — not a commit timestamp. Greptile re-reviews by editing its summary in place, so timestamps cannot observe a completed re-review; a timestamp proxy re-triggers commits Greptile has already approved. Timestamps survive only as the fallback when no marker parses.

Every fetch failure fails safe by posting rather than assuming satisfaction.

Fitting it to this repo

  • Step 2i (final mandatory re-trigger) added; Return result renumbered to 2j. The gate is designed to run twice, and its own header names Step 2i. That header doubles as the sentinel the shared test suite locates the gate by, so aligning this repo's numbering keeps one identical suite proving every fleet copy. I first tried the reverse — rewording the sentinel to say "Step 2h.1" to match the existing layout — and it immediately broke the suite's locator, which is the right outcome: that line is a fleet contract, not prose.
  • The 50-trigger hard cap is preserved, and Step 2g's prose now says explicitly that the gate is not exempt from it — if the cap is spent, follow the cap's instructions instead of running the gate.
  • The Step 2g prose was corrected. It still told the reader to "skip the actual trigger only if Greptile already reacted to your most recent reply" — the precise rule the gate exists to remove, and a direct contradiction of the code beneath it.

Verification

test_sweep_greptile_gate.py ported alongside. It extracts the real fenced block from SKILL.md — no copy — and runs it against fixture PR state behind a stub gh, under both bash and zsh (zsh because it is the maintainers' interactive shell, and a /sweep guard in a sibling repo once silently passed under zsh while failing its job, since zsh does not word-split an unquoted parameter).

Three of its six cases are mutation tests: they revert the gate to the pre-fix behaviour — remove the marker lookup, break the SHA regex, ignore a marker mismatch — and require the regression assertion to fail. Without them that assertion could pass with and against the bug it names.

6/6 on this branch. Wired into CI as sweep-gate.yml, path-filtered to the skill + suite — without it nothing ever executes this gate: it lives in Markdown, so it is shipped, load-bearing and otherwise entirely unproven.

Part of the fleet-wide sweep tracked by optave/data-retrieval-storage-svc#1021.

…on trigger gate

Step 2g decided "is Greptile satisfied?" by checking for a positive reaction on
THE MOST RECENT NON-GREPTILE COMMENT - i.e. on one of your own replies. A 👍 on a
reply is not a re-review of your code. Under that rule the re-trigger is skipped
while the fix sits un-reviewed, which is how a sweep reports a PR as converged
when Greptile has never seen the change.

Transplant the four-condition gate the fleet standardised on. Satisfied now
requires ALL of: an @greptileai TRIGGER comment exists and really mentions
Greptile (a literal inside a code span, fenced block, indented block or HTML
comment notifies nobody and is not a trigger); Greptile reacted positively TO THAT
TRIGGER; Greptile posted nothing since; and Greptile has reviewed the CURRENT head,
established from its own `Last reviewed commit` marker rather than a commit
timestamp - Greptile re-reviews by EDITING its summary IN PLACE, so timestamps
cannot observe a completed re-review. Every fetch failure fails safe by posting.

Add Step 2i (final mandatory re-trigger) and renumber Return result to 2j, matching
the fleet layout. The gate is meant to run twice and its own header names Step 2i;
that header doubles as the sentinel the shared test suite locates it by, so aligning
this repo's numbering keeps one identical suite proving every copy - the alternative,
rewording the sentinel per repo, would fork the contract.

The 50-trigger cap is preserved and now stated where it matters: the gate is not
exempt from it, and Step 2g's prose says so. That prose previously taught the exact
rule the gate replaces ("skip the actual trigger only if Greptile already reacted to
your most recent reply"), contradicting the code beneath it.

Verified with test_sweep_greptile_gate.py, ported alongside: it extracts the real
fenced block from SKILL.md and runs it behind a stub gh under both bash and zsh.
Three of its six cases are mutation tests that re-introduce the pre-fix behaviour
and require the regression assertion to fail. 6/6. Wired via sweep-gate.yml, since
a gate that lives in Markdown is otherwise never executed by anything.

Refs: optave/data-retrieval-storage-svc#1021, optave/data-retrieval-storage-svc#930
@greptile-apps

greptile-apps Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR replaces reply-reaction satisfaction with a four-condition Greptile trigger gate and adds a cross-shell fixture suite plus path-filtered CI. Two fetch-error paths still fail to uphold the gate’s stated fail-safe behavior.

  • Classifies genuine trigger mentions while excluding code and comment literals.
  • Uses Greptile’s reviewed-commit marker before falling back to timestamps.
  • Tests the embedded shell under bash and zsh, including mutation cases.
  • Adds a dedicated GitHub Actions workflow.

Confidence Score: 3/5

The PR should not merge until failed comment fetches consistently attempt the mandatory trigger rather than aborting or permitting timestamp-based satisfaction.

The new gate can terminate without posting on its first fetch failure and can silently treat a later marker-fetch failure as an absent marker, allowing the documented inexact timestamp proxy to skip review of the current head.

Files Needing Attention: .claude/skills/sweep/SKILL.md

Important Files Changed

Filename Overview
.claude/skills/sweep/SKILL.md Adds the four-condition gate and mandatory final invocation, but two comment-fetch failures can abort or incorrectly fall back instead of safely posting.
.github/scripts/test_sweep_greptile_gate.py Adds substantive cross-shell and mutation coverage, though it does not isolate the marker-fetch failure that permits a false skip.
.github/workflows/sweep-gate.yml Adds an appropriately scoped workflow that installs zsh and executes the standalone gate suite.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Run Step 2g or Step 2i gate] --> B[Fetch trigger comments]
  B -->|Failure| X[Exit without posting]
  B -->|Success| C[Check trigger reaction and later comments]
  C --> D[Fetch current head]
  D --> E[Fetch reviewed-commit markers]
  E -->|Marker matches head| F[Skip trigger if all conditions hold]
  E -->|No marker or fetch failure| G[Timestamp fallback]
  G -->|Timestamp not newer| F
  G -->|Timestamp newer or unavailable| H[Post @greptileai]
Loading

Fix All in Greploop

Fix All in Claude Code

Reviews (1): Last reviewed commit: "fix(sweep): replace the reply-reaction s..." | Re-trigger Greptile

Comment on lines +382 to +384
trigger_candidates=$(gh api repos/<repo>/issues/<number>/comments --paginate \
--jq '.[] | select(.user.login != "greptile-apps[bot]") | "\(.id)\t\(.created_at)\t\((.body // "") | @json)"') \
|| { echo "FATAL: could not fetch trigger comments — aborting gate"; exit 1; }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Trigger fetch aborts fail-safe gate

When the issue-comments request fails during Step 2g or mandatory Step 2i, this branch exits before attempting to post @greptileai, causing the sweep to stop without completing the required final review trigger.

Fix in Claude Code

Comment on lines +562 to +565
reviewed_shas=$(gh api repos/<repo>/issues/<number>/comments --paginate \
--jq '.[] | select(.user.login == "greptile-apps[bot]") | .body' 2>/dev/null \
| grep -o 'Last reviewed commit:.*/commit/[0-9a-fA-F]\{40\}' \
| grep -o '[0-9a-fA-F]\{40\}$' | tr 'A-Z' 'a-z')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Marker fetch failure permits false skip

When this issue-comments request fails after the initial fetch succeeded, the pipeline silently produces an empty reviewed_shas and enters the timestamp fallback. If the commit timestamp is no later than the trigger, the gate treats the head as reviewed and skips the mandatory trigger even though Greptile's current-head marker was never retrieved.

Fix in Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant