Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions .claude-jobs/review-issue.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,17 @@ schedule: "*/5 * * * *"
enabled: true
auth: subscription

# The 1800 passed to find-review-candidate.sh is how long (in seconds) a
# `review in progress` label must sit before the PR is taken back from a dead
# run: `timeout` below plus 5 minutes. Keep it above `timeout`, and keep both
# call sites in step with each other and with the script's default and header.
preflight:
run: .claude/skills/review-issue/scripts/find-review-candidate.sh > /dev/null
run: .claude/skills/review-issue/scripts/find-review-candidate.sh 1800 > /dev/null
timeout: 30s

claude:
prompt_cmd: |
pr=$(.claude/skills/review-issue/scripts/find-review-candidate.sh)
pr=$(.claude/skills/review-issue/scripts/find-review-candidate.sh 1800)
echo "/review-issue $pr"

append_system_prompt: |
Expand All @@ -24,6 +28,18 @@ claude:
to other skills like next-deployer. If the skill suggests merging or
handing off, skip those steps and exit.

Never wait for CI. This run is killed at its timeout, and a wait throws
the finished review away. Right before deciding, run
`.claude/skills/review-issue/scripts/pending-checks.sh <pr>` once,
instead of reading `gh pr checks`; it prints how many check runs have
not completed. If that's more than 0, or the script fails without
printing a number, post nothing, set the label back to `needs review`
with set-review-label.sh, and stop (skill step 4,
"Cron mode"). If it prints 0, decide normally: zero check runs, and
checks that finished as failed, skipped or cancelled, are not pending.
No `gh pr checks --watch`, sleep or poll loops, background watchers,
Monitor, or ScheduleWakeup.

allowed_tools:
- "Bash(gh pr view*)"
- "Bash(gh pr list*)"
Expand All @@ -35,6 +51,7 @@ claude:
- "Bash(.claude/skills/review-issue/scripts/set-review-label.sh*)"
- "Bash(.claude/skills/review-issue/scripts/ensure-review-labels.sh*)"
- "Bash(.claude/skills/review-issue/scripts/post-review.sh*)"
- "Bash(.claude/skills/review-issue/scripts/pending-checks.sh*)"
# Write is allowed so the agent can produce the review-body file that
# post-review.sh consumes (avoids the backtick-escaping artifact that
# happens when markdown is shoved into `gh pr review --body "..."`).
Expand All @@ -48,10 +65,18 @@ claude:
- "Read"
- "Grep"

# A one-shot cron run can't be woken up later or wait out CI, so these can
# only strand the review (#178).
disallowed_tools:
- "ScheduleWakeup"
- "Monitor"

permission_mode: auto

cwd: "."
timeout: 10m
# A full review of a medium-sized PR, including the second-opinion reviewer
# subagent, takes well over 10 minutes (#178).
timeout: 25m

logging:
retention_days: 30
4 changes: 4 additions & 0 deletions .claude/skills/review-issue/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ Move the PR into the "review in progress" state. This single call removes any of
.claude/skills/review-issue/scripts/set-review-label.sh <pr-number> "review in progress"
```

Run it even when the PR already shows `review in progress`. That means an earlier review died before posting a verdict and the cron took the PR back. The script re-applies the label so this attempt is recorded, which is how the cron knows not to take the PR back a second time.

### 2. Establish acceptance criteria

- **Explicit** — issue lists them → use those
Expand All @@ -73,6 +75,8 @@ Skill(skill="superpowers:requesting-code-review")

### 4. Decide

> **Cron mode:** when running non-interactively, never wait for CI. Run `.claude/skills/review-issue/scripts/pending-checks.sh <pr-number>` once, right before deciding; use it instead of reading `gh pr checks`. It prints how many check runs on the head commit have not completed. If that's more than 0, or the script fails without printing a number, post nothing, move the label back with `.claude/skills/review-issue/scripts/set-review-label.sh <pr-number> "needs review"`, and stop. The cron skips PRs with pending checks (using the same script), so the next run picks this one up after CI finishes. If it prints 0, nothing is pending, so decide normally. That includes a PR with no check runs at all (`gh pr checks` then says "no checks reported") and checks that finished as failed, skipped or cancelled. A failed check is a review finding, not a reason to wait. Don't use `gh pr checks --watch`, sleep or poll loops, background watchers, `Monitor`, or `ScheduleWakeup`: the run is killed at the job's timeout and the finished review is lost.

**If ANY blocking issue is found OR any acceptance criterion is unmet:**

Build the review body using GitHub-flavored markdown. Blocking items appear at the top; non-blocking observations and nits are hidden in collapsible `<details>` blocks. Omit any section that has no items.
Expand Down
125 changes: 105 additions & 20 deletions .claude/skills/review-issue/scripts/find-review-candidate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,58 @@
# - labeled `needs review`, or
# - labeled `changes requested` AND at least one commit has been pushed to
# the PR head after the `changes requested` label was applied (i.e., the
# author has responded to feedback and the PR is ready for re-review).
# author has responded to feedback and the PR is ready for re-review), or
# - labeled `review in progress` AND the label was applied more than
# stale-after seconds ago (the run that applied it died without posting a
# verdict). Taken back at most once per head commit, so a review that dies
# every time isn't retried forever; a new commit or a reset to
# `needs review` re-arms it.
#
# ...and every check run on the PR's head commit has completed (see
# pending-checks.sh). A pending PR is deferred, not dropped: the script moves
# on to the next PR, and the deferred one qualifies on the first tick after its
# checks finish. A review can't pass without green CI, so a run started
# earlier could only wait out its timeout.
#
# Skips PRs labeled `first pass reviewed` or `approved` — those belong to the
# final-review cron or are already done.
#
# Exits 0 with the PR number on stdout when a candidate is found.
# Exits 1 silently when nothing is ready.
# Exits 1 when nothing is ready. Deferred and capped PRs are noted on stderr.
#
# Usage: find-review-candidate.sh [stale-after-seconds]
# Defaults to 1800 (30 minutes): the review-issue job's 25-minute `timeout`
# plus 5 minutes. The cron's overlap lock already keeps a live review-issue
# run from being taken back; the threshold protects interactive
# `/review-issue` sessions, so keep it above the job's `timeout`.

set -euo pipefail

source "$(git rev-parse --show-toplevel)/scripts/gh-pin-account.sh"

stale_after="${1:-1800}"
if ! [[ "$stale_after" =~ ^[0-9]+$ ]]; then
echo "usage: $0 [stale-after-seconds]" >&2
exit 2
fi

repo="normalled/apijack"
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
now=$(date -u +%s)

# Every `labeled` event on PR $1 as "<created_at><TAB><label>", oldest first.
# Paginated: the events API returns oldest first, so a long-lived PR's recent
# label events aren't on the first page. --jq runs once per page, so emit one
# line per event and aggregate afterwards.
labeled_events() {
gh api --paginate "repos/$repo/issues/$1/events?per_page=100" \
--jq '.[] | select(.event == "labeled") | "\(.created_at)\t\(.label.name)"'
}

# Timestamps of label $2 in labeled_events output $1, oldest first.
label_times() {
awk -F '\t' -v label="$2" '$2 == label { print $1 }' <<<"$1" | sort
}

# One API call: list candidate PRs with labels + head SHA.
prs=$(gh pr list --repo "$repo" \
Expand All @@ -35,31 +74,77 @@ while read -r pr_json; do
n=$(jq -r '.number' <<<"$pr_json")
labels=$(jq -r '.labels[]' <<<"$pr_json")
head=$(jq -r '.head' <<<"$pr_json")
taken_back_from=""

# `needs review` is unconditionally eligible.
if grep -qxF "needs review" <<<"$labels"; then
found="$n"
break
# `needs review` is eligible as soon as its checks settle.
:
elif grep -qxF "changes requested" <<<"$labels"; then
# `changes requested` is eligible only when a commit lands after the label.
events=$(labeled_events "$n")
labeled_at=$(label_times "$events" "changes requested" | tail -n 1)
[ -z "$labeled_at" ] && continue

head_at=$(gh api "repos/$repo/commits/$head" --jq '.commit.committer.date // empty')
[ -z "$head_at" ] && continue

labeled_ts=$(date -u -d "$labeled_at" +%s)
head_ts=$(date -u -d "$head_at" +%s)
[ "$head_ts" -gt "$labeled_ts" ] || continue
elif grep -qxF "review in progress" <<<"$labels"; then
# `review in progress` is eligible once stale (see the header). Every
# review attempt leaves a `labeled` event (set-review-label.sh re-applies
# this label even when it's present), so attempts are counted from the
# events after the head commit or the last reset to `needs review`,
# whichever is later.
events=$(labeled_events "$n")
attempts_at=$(label_times "$events" "review in progress")
[ -z "$attempts_at" ] && continue

labeled_at=$(tail -n 1 <<<"$attempts_at")
labeled_ts=$(date -u -d "$labeled_at" +%s)
[ $(( now - labeled_ts )) -lt "$stale_after" ] && continue

head_at=$(gh api "repos/$repo/commits/$head" --jq '.commit.committer.date // empty')
[ -z "$head_at" ] && continue

since_ts=$(date -u -d "$head_at" +%s)
reset_at=$(label_times "$events" "needs review" | tail -n 1)
if [ -n "$reset_at" ]; then
reset_ts=$(date -u -d "$reset_at" +%s)
if [ "$reset_ts" -gt "$since_ts" ]; then
since_ts="$reset_ts"
fi
fi

attempts=0
while read -r at; do
if [ "$(date -u -d "$at" +%s)" -gt "$since_ts" ]; then
attempts=$(( attempts + 1 ))
fi
done <<<"$attempts_at"
if [ "$attempts" -ge 2 ]; then
echo "PR #$n: 'review in progress' stalled again after a take-back on ${head:0:7};" \
"not retrying until a new commit or a reset to 'needs review'" >&2
continue
fi
taken_back_from="$labeled_at"
else
continue
fi

# `changes requested` is eligible only when a commit lands after the label.
if ! grep -qxF "changes requested" <<<"$labels"; then
pending=$("$script_dir/pending-checks.sh" "$n")
if [ "$pending" -gt 0 ]; then
echo "PR #$n: deferred, $pending check run(s) still pending" >&2
continue
fi
labeled_at=$(gh api "repos/$repo/issues/$n/events" \
--jq 'map(select(.event == "labeled" and .label.name == "changes requested"))
| sort_by(.created_at) | reverse | .[0].created_at // empty')
[ -z "$labeled_at" ] && continue

head_at=$(gh api "repos/$repo/commits/$head" --jq '.commit.committer.date // empty')
[ -z "$head_at" ] && continue

labeled_ts=$(date -u -d "$labeled_at" +%s)
head_ts=$(date -u -d "$head_at" +%s)
if [ "$head_ts" -gt "$labeled_ts" ]; then
found="$n"
break

if [ -n "$taken_back_from" ]; then
echo "PR #$n: taking back 'review in progress' left by a dead run (applied $taken_back_from)" >&2
fi

found="$n"
break
done < <(jq -c '.' <<<"$prs" 2>/dev/null || echo "$prs")

[ -n "$found" ] || exit 1
Expand Down
32 changes: 32 additions & 0 deletions .claude/skills/review-issue/scripts/pending-checks.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/usr/bin/env bash
# Print how many check runs on a PR's head commit have not completed yet.
#
# 0 means nothing is pending: every check run has completed, whatever its
# conclusion (failed, skipped and cancelled included), or the head has no check
# runs at all, which path-filtered workflows legitimately produce.
#
# The review cron's candidate query (find-review-candidate.sh) and the
# review-issue skill's cron-mode CI check both use this, so the two can't
# disagree about whether CI is still running.
#
# Usage: pending-checks.sh <pr-number>

set -euo pipefail

source "$(git rev-parse --show-toplevel)/scripts/gh-pin-account.sh"

if [ $# -ne 1 ]; then
echo "usage: $0 <pr-number>" >&2
exit 2
fi

pr="$1"
repo="normalled/apijack"

head=$(gh pr view "$pr" --repo "$repo" --json headRefOid --jq '.headRefOid')

# --jq runs once per page under --paginate, so emit a line per pending run and
# count them here.
count=$(gh api --paginate "repos/$repo/commits/$head/check-runs?per_page=100" \
--jq '.check_runs[] | select(.status != "completed") | .id' | wc -l)
echo $(( count ))
10 changes: 10 additions & 0 deletions .claude/skills/review-issue/scripts/set-review-label.sh
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,16 @@ for l in "${review_labels[@]}"; do
gh api -X DELETE "repos/$repo/issues/$pr/labels/$encoded" >/dev/null
done

# `review in progress` is re-applied even when the PR already has it (adding a
# label that's present is a no-op on GitHub), so every review attempt leaves a
# fresh `labeled` event. find-review-candidate.sh times stale reviews and caps
# take-backs by those events.
if [ "$target" = "review in progress" ] && grep -qxF "$target" <<<"$current"; then
encoded=$(jq -rn --arg s "$target" '$s|@uri')
gh api -X DELETE "repos/$repo/issues/$pr/labels/$encoded" >/dev/null
current=$(grep -vxF "$target" <<<"$current" || true)
fi

if ! grep -qxF "$target" <<<"$current"; then
gh api -X POST "repos/$repo/issues/$pr/labels" -f "labels[]=$target" >/dev/null
fi
Expand Down
Loading
Loading