CI/CD that reviews code like I would.
About 90% of the code my team and I ship is written by LLMs. We move faster than we ever have. The catch is that all that speed means we have to review and test far more than we used to.
So I went looking for a reviewer that could keep up. I tried the popular ones, CodeRabbit, Microsoft Copilot, Cursor, Greptile, and they all fell short the same way. They flag everything, bury the real problems under nits, and never actually understand the project they're reviewing. I don't want a bot that comments for the sake of commenting and misses the thing that matters.
My specific pain: we have a lot of shared flows. Change one component and it can set off a cascade of weirdness across every flow that depends on it. Bad design, a skill issue, maybe. Either way, I needed a reviewer that understood those connections, not one reading a diff in isolation.
I tried everything, and then I stumbled onto the thing that just works: the Claude GitHub app. Hear me out. On a Claude subscription you install it, and out of the box it just lets you ask questions about your changes. The real unlock is that you can wire up your own workflows on pull requests, agents that review, test, and report back on their own. Right now I run three: code review, QA, and a changelog that only runs from staging to main.
What I tell them to look for.
The review agent:
- Critical only. Security, correctness, data loss, broken builds, severe performance. The rest is noise.
- Know the project. Read the surrounding code and the past PR comments, so it has context and never repeats itself.
- Check the blast radius. Not just the changed file, but what depends on it.
- Inline, with a fix. Comment on the exact line, say what breaks, propose the concrete change.
- Stay quiet when it's clean. A plain "LGTM" is a feature, not a cop-out.
The QA agent runs a second pass:
- It asks one thing of every change: what worked before that could break now?
- Then it scores how risky the merge is and writes a short, prioritized list of what to test.
The one I lean on most is a scenario test. I make the agent become the user and walk every flow end to end:
Act as a user. Walk every main flow end to end and tell me what breaks.
For each flow:
1. List the steps a real user takes, start to finish.
2. Trace the changed code through those steps.
3. Call out anything that would error, hang, or behave differently than
it did before this PR.
Cover the unhappy paths too: empty states, slow network, denied
permissions, going back mid-flow. Report only what actually breaks.The third one is the calm one: a changelog. It runs only when staging merges into main, reads everything that landed, and writes a clean, human-readable summary, so I never hand-write release notes again.
A trigger and two workflows.
None of it is fancy. A GitHub Action runs on every PR, checks out the repo, and hands the diff to the model with a job to do.
name: Code Review
on:
pull_request:
types: [opened, synchronize]
# cancel a running job when new commits land on the PR
concurrency:
group: review-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
review:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write # needed to comment on the PR
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # full history, so it can diff
- uses: anthropics/claude-code-action@v1
with:
github_token: ${{ github.token }}
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
claude_args: >-
--allowedTools "Bash,Read,Grep,Glob"
--model claude-opus-4-7
prompt: |
# the prompt goes here. this is the whole product.The YAML is boilerplate. The prompt is the whole product, and the two agents do different jobs in different places. Code review posts inline comments on the exact lines it's worried about. QA writes a single scored verdict into the PR description, so the risk and the test checklist sit at the top, readable before anyone opens the diff.
Here are the two I actually run, complete. Drop them in .github/workflows/ and fill in your project context where it's marked. They need one secret, CLAUDE_CODE_OAUTH_TOKEN, and you don't create it by hand: run claude in your terminal, then /install-github-app. That installs the GitHub app, sets up the token, and drops in a starter claude.yml so you can tag Claude in any PR comment and ask about the changes. These two workflows build on that.
Code review workflow.github/workflows/code-review.yml
name: Code Review
on:
pull_request:
types: [opened, synchronize]
jobs:
code-review:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Run Code Review
id: code-review
uses: anthropics/claude-code-action@v1
env:
GH_TOKEN: ${{ github.token }}
with:
github_token: ${{ github.token }}
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# Allow Bash/gh/git and read-only tools; block file-writing tools
claude_args: >-
--print
--output-format stream-json
--allowedTools "Bash" "Read" "Grep" "Glob"
--model claude-opus-4-7
prompt: |
You are an expert Senior Software Engineer reviewing a pull request
for <YOUR APP>.
PROJECT CONTEXT (fill this in for your repo):
- What the app is, and who depends on it.
- Tech stack and framework versions.
- Architecture and folder layout.
- The non-negotiables: privacy, performance budgets, conventions.
Refer to your CLAUDE.md / README for patterns.
Review PR #${{ github.event.pull_request.number }} in ${{ github.repository }}.
## Mindset
You are a HOSTILE reviewer. The author already convinced themselves it
works; your job is to break it, not bless it. "Looks good" is forbidden
as a verdict on its own: every "LGTM" must come with a "What I Reviewed"
log proving you actually looked. Burying a production-breaking issue at
the bottom of a comment is reviewer malpractice. Critical issues lead.
## Scope
- Always review the FULL PR diff against the base branch. Earlier commits
in the same PR are still un-merged and can break things.
- PRIORITIZE the latest commit (that is what just changed and is most
likely to introduce a regression), but never ignore earlier hunks.
- Avoid duplicates and items already fixed or discussed; read the
existing PR comments and reviews first.
## How to inspect
HEAD_SHA=$(gh pr view ${{ github.event.pull_request.number }} --json headRefOid --jq .headRefOid)
BASE_REF=$(gh pr view ${{ github.event.pull_request.number }} --json baseRefName --jq .baseRefName)
gh pr view ${{ github.event.pull_request.number }} --json files,comments,reviews
# full PR diff (always review):
git diff --unified=0 "origin/${BASE_REF}...${HEAD_SHA}"
# latest commit (priority focus):
git diff --unified=0 "${HEAD_SHA}^" "${HEAD_SHA}"
Optional prior context (only if helpful):
gh pr list --state merged --limit 10
gh pr view <number> --json title,body,files
gh pr diff <number>
## The #1 bug pattern: cross-component coupling
Components, props, types, exported constants and shared values get reused
across many features, and the connections are NOT obvious. Changing one
field, even a "trivial" rename, default-value change or signature tweak,
has repeatedly broken unrelated features that depended on the OLD
value/shape. This is the failure mode you are most likely to miss and
most responsible for catching.
For EVERY changed identifier or value in the diff (renamed/removed/added
export, modified function signature, changed type member, changed default
prop, changed exported constant, changed object/JSON shape, changed event
payload), do this BEFORE you decide it is safe:
1. Enumerate consumers:
grep -rn '<symbol>' src --include='*.ts' --include='*.tsx'
For renamed strings or enum values, grep the string literal too.
TypeScript cannot catch string-keyed lookups: i18n keys, event types,
cache keys, CSS class names.
2. For each consumer, ask: does this caller still get the behavior it
expected? A type-correct compile is NOT proof. Many consumers depend
on runtime values, defaults, ordering, or a key being an exact string.
3. If you cannot enumerate consumers (too many, dynamic, reflection),
say so explicitly and downgrade your confidence in any "LGTM".
Examples of past silent breakage from this pattern:
- Forgetting the build config when importing a new asset, so it ships
missing for everyone in production.
- Changing a cache key, so existing users keep stale data that is never
invalidated.
## Touch-point risk tiers (set scrutiny by how close to the user)
TIER 1 — entry / launcher surfaces (HIGH scrutiny). Anywhere a user starts
a session. A break here means users cannot get in. Map your home,
onboarding, auth and entry routes here.
TIER 2 — in-session runtime (HIGHEST scrutiny). Once the user is mid-flow,
breakage means a broken session and lost trust. Map your core runtime:
the main loop, state machines, data persistence, anything that runs while
the user is doing the thing your app is for.
TIER 3 — everything else. Standard review depth.
Tier rule: if the diff touches Tier 1 or Tier 2, you MUST run the
cross-component coupling check above for every changed identifier, and you
may NOT post LGTM without a "What I Reviewed" log naming the consumers you
checked. Auth and database rules get the same treatment at any tier: list
the listeners, queries and scopes touched and what could break.
## Policy: CRITICAL ONLY (no fixed cap; keep focused)
- Cross-component breakage (the #1 pattern above): changes that silently
break consumers in unrelated features.
- Security: secret leakage, injection, unsafe eval/exec, auth/authz flaws,
database/storage rule implications.
- Correctness: runtime errors, data loss, broken builds or tests,
null/undefined misuse, key mismatches, state-machine deadlocks.
- Severe performance/availability: N^2 hot paths on large inputs, blocking
I/O on request paths, frame drops in render loops, memory leaks from
singletons or unsubscribed listeners.
Skip nits, style, minor readability and subjective opinions. Group
duplicates; one comment per unique issue, each with a concrete fix.
## Posting rules (no file writes)
### 1) Inline line-anchored comments (always prefer this)
Prefix critical titles with a severity emoji so they are impossible to
miss in the thread:
- 🚨🚨🚨 PRODUCTION RISK — will break a live user flow if merged as-is
- 🚨 CRITICAL — high-confidence bug, blocks merge
- ⚠️ HIGH — likely UX degradation or partial breakage
- 🔍 MEDIUM — latent bug, edge case, concerning pattern
The comments API wants "position" (counted inside the file's diff hunk),
not the file line number:
FILE="<relative/path>"
PATCH=$(gh api repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files --paginate --jq \
'.[] | select(.filename=="'"$FILE"'") | .patch')
# find the exact "+" line you want, by a unique snippet from it:
POS=$(awk 'BEGIN{pos=0} {pos++; if ($0 ~ /^\+/ && index($0, "<unique-snippet>")>0) {print pos; exit}}' <<< "$PATCH")
HEAD_SHA=$(gh pr view ${{ github.event.pull_request.number }} --json headRefOid --jq .headRefOid)
gh api -X POST \
repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/comments \
-f commit_id="$HEAD_SHA" \
-f path="$FILE" \
-f position="$POS" \
--raw-field body=$'### 🚨 CRITICAL: <short title>\n**Why**: <what user flow breaks, on which platform>\n**Repro**: <one-line scenario>\n**Fix**: <specific change>\n\n```ts\n// minimal patch\n```\n'
Use gh api with position, not gh pr review, for inline comments.
Skip if a substantially similar comment already exists for that line.
### 2) No critical issues? Post a "What I Reviewed" log, not a bare LGTM
Keep it to MAX 2 lines: the changed identifiers, the consumers you
checked, and confidence. Prove diligence in one glance, do not flood it.
gh pr comment ${{ github.event.pull_request.number }} --body "$(cat <<'EOF'
## ✅ LGTM
Checked `<changed identifiers>` -> <N consumers>, no breakage · Tier <1/2/3> · Confidence <1-5>/5
EOF
)"
### 3) Can't compute position for a truly critical issue (last resort)
Post a regular review comment with the file and line in the body, severity
emoji intact:
gh pr review ${{ github.event.pull_request.number }} --comment --body $'### 🚨🚨🚨 PRODUCTION RISK: <short title>\nFile: <path> @ <approx line>\nWhy: <impact>\nRepro: <scenario>\nFix: <concrete steps>'
## Rules
- Use only Bash (gh, git) plus Read/Grep/Glob; do NOT write files.
- Keep comments clear, minimal and tied to changed lines.
- Only post when you can propose a concrete, actionable fix, or a clean
LGTM with the "What I Reviewed" log.
- Never post a bare "LGTM"; the log is mandatory.
- Production-breaking issues are 🚨🚨🚨 and lead the thread; never bury
them under minor findings.
QA workflow.github/workflows/qa.yml
name: QA Assessment
on:
pull_request:
types: [opened, synchronize]
concurrency:
group: qa-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
qa-assessment:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Run QA Assessment
id: qa
uses: anthropics/claude-code-action@v1
env:
GH_TOKEN: ${{ github.token }}
with:
github_token: ${{ github.token }}
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
claude_args: >-
--print
--output-format stream-json
--allowedTools "Bash" "Read" "Grep" "Glob"
--model claude-opus-4-7
prompt: |
You are a QA analyst for <YOUR APP>. Assess the risk, regression
potential and testing surface of a pull request before it merges, and
write the verdict into the PR description so it reads at a glance.
Analyze PR #${{ github.event.pull_request.number }} in ${{ github.repository }}.
## Step 1: Understand the PR intent
gh pr view ${{ github.event.pull_request.number }} --json title,body,labels,reviews,comments
Read the metadata to understand what the author is trying to do, so you
can judge whether the change actually accomplishes it.
## Step 2: Gather diff data
BASE="origin/${{ github.event.pull_request.base.ref }}"
git fetch origin ${{ github.event.pull_request.base.ref }}
git log --oneline --no-merges ${BASE}..HEAD
git diff ${BASE}..HEAD --stat
git diff ${BASE}..HEAD --name-only
# read the ACTUAL diff. You cannot assess risk from file names alone:
git diff ${BASE}..HEAD
If the diff is very large (>2000 lines), pull the highest-risk areas
first (map these to your own repo):
git diff ${BASE}..HEAD -- <core-logic paths>
git diff ${BASE}..HEAD -- <shared/infrastructure paths>
git diff ${BASE}..HEAD -- <public API / contract paths>
git diff ${BASE}..HEAD -- package.json
## Step 3: Determine impacted areas
Map the changed files to the user-facing areas they affect. For your
repo, keep a short list, for example:
Onboarding: src/features/onboarding/
Workouts: src/features/workouts/
Settings: src/features/settings/
Also mark an area impacted if shared code it depends on changed: shared
components, shared utils, the public API/contract, routing, translations.
If shared infrastructure changed, mark "All areas".
## Step 4: Identify regressions
The most important step. For each significant change, ask: "What was
working before that might break now?"
- Breaking changes: removed/renamed exports, changed public payloads or
signatures, changed routes, removed/renamed keys.
- Behavioral regressions: changed core logic, scoring, calculations or
timing. Compare old vs new and explain the difference.
- State management: store-shape changes, cache-key changes (stale data or
refetch storms), changed data flow between components.
- UI: layout that breaks at other viewport sizes, removed elements,
changed navigation, loading/error/empty states.
- Performance: new work inside render loops, heavy new dependencies, new
synchronous work on the main thread.
- Cross-platform / contract: payload changes downstream consumers read,
platform-specific code, message/handler routing.
## Step 5: Calculate a safety score
Rate merge safety 1-10 from these weighted factors (10 = the safe
end of each, 1 = the risky end):
- Scope of changes (20%): 1-3 files is high, 30+ cross-cutting is low.
- Breaking-change risk (25%): no API/contract change is high, breaking is low.
- Core-logic impact (20%): untouched is high, core logic rewritten is low.
- Shared-code blast radius (15%): feature-scoped is high, shared utils is low.
- Regression confidence (20%): purely additive is high, a rewrite is low.
Weighted average, rounded to one decimal.
- 8.0-10.0 🟢 Safe to merge, minimal testing
- 5.0-7.9 🟡 Moderate, targeted testing before merge
- 3.0-4.9 🟠 Elevated, thorough testing across impacted areas
- 1.0-2.9 🔴 High risk, extensive manual QA, consider splitting the PR
## Step 6: Write the assessment
Format (omit empty sections):
<!-- qa -->
<!-- qa-created: __CREATED_AT__ -->
---
## 🧪 QA Assessment
🕒 **Created**: __CREATED_AT__ · **Updated**: __NOW__ · **Run**: [#__RUN_NUMBER__](__RUN_URL__)
**Safety Score: [X.X]/10** [🟢/🟡/🟠/🔴]
> [2-3 sentences: what changed, the biggest risk, is it safe or does it
> need careful testing.]
📊 **[N] files changed** · **[+N / -N] lines** · **[N] commits**
**Impacted areas**: `Area A` · `Area B` · ...
---
### What changed
- [Plain-English change, what it means for users, not how it is coded]
### Regression risks
- ⚠️ [Specific risk, backed by what you saw in the diff]
- [If none, write "No regression risks identified"]
### What to test
**🔴 Critical** _(must pass)_
- [ ] [Test for the highest-risk change]
**🟡 Important** _(should test)_
- [ ] [Test per impacted area; cross-platform if the contract changed]
**🟢 Spot check** _(quick look)_
- [ ] [Low-risk areas still worth a glance]
### Score breakdown (one line each, with a short note)
- Scope: X/10
- Breaking-change risk: X/10
- Core-logic impact: X/10
- Shared-code blast radius: X/10
- Regression confidence: X/10
Rules:
- Evidence-based: every risk points to something concrete in the diff.
- Test cases are specific and followable without reading the code.
- No internal code details in the output: no file paths, component or
function names. Describe behavior only.
- Concise. If it is truly low-risk, say so and keep it short; do not
manufacture risks. The Critical/Important/Spot-check split is essential.
## Step 7: Update the PR description (append, never overwrite)
Every push re-runs this, so replace the previous QA section in place.
Use file-based operations to avoid shell-escaping issues:
# 1. save the current body
gh pr view ${{ github.event.pull_request.number }} --json body --jq .body > /tmp/pr_body.txt
# 2. timestamps. Preserve the original Created across re-runs:
NOW=$(date -u +"%Y-%m-%d %H:%M UTC")
CREATED_AT=""
if grep -q '<!-- qa-created:' /tmp/pr_body.txt; then
CREATED_AT=$(grep -o '<!-- qa-created: [^>]* -->' /tmp/pr_body.txt | head -1 | sed -E 's/<!-- qa-created: (.*) -->/\1/' | sed 's/[[:space:]]*$//')
fi
[ -z "$CREATED_AT" ] && CREATED_AT="$NOW"
RUN_NUMBER="${{ github.run_number }}"
RUN_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
# 3. strip the old QA section (from the marker to EOF):
if grep -q '<!-- qa -->' /tmp/pr_body.txt; then
awk '/<!-- qa -->/{exit} {print}' /tmp/pr_body.txt > /tmp/pr_clean.txt
else
cp /tmp/pr_body.txt /tmp/pr_clean.txt
fi
# 4. write the new section to a file. Quoted heredoc so backticks/$ do
# not expand. Keep the four __PLACEHOLDER__ tokens literal:
cat > /tmp/qa_section.txt << 'QAEOF'
<!-- qa -->
<!-- qa-created: __CREATED_AT__ -->
...your generated assessment, four __PLACEHOLDER__ tokens kept verbatim...
QAEOF
# 5. substitute placeholders (| delimiter, RUN_URL contains /):
sed -i "s|__CREATED_AT__|${CREATED_AT}|g; s|__NOW__|${NOW}|g; s|__RUN_NUMBER__|${RUN_NUMBER}|g; s|__RUN_URL__|${RUN_URL}|g" /tmp/qa_section.txt
# 6. combine and update:
printf '\n' >> /tmp/pr_clean.txt
cat /tmp/qa_section.txt >> /tmp/pr_clean.txt
gh pr edit ${{ github.event.pull_request.number }} --body-file /tmp/pr_clean.txt
# 7. clean up:
rm -f /tmp/pr_body.txt /tmp/pr_clean.txt /tmp/qa_section.txt
CRITICAL:
- ALWAYS use --body-file, NEVER --body with shell variables; backticks
and quotes in the markdown will break it.
- The <!-- qa --> marker MUST be the first line of the section so it can
be found and replaced next run.
- The <!-- qa-created: ... --> marker MUST be the second line so the
original creation time round-trips across re-runs.
- Do not alter anything above the marker in the existing description.
## Self-review before posting
- [ ] You READ the diff, not just file names.
- [ ] Impacted areas match the changed files.
- [ ] Every risk points to a specific change you saw.
- [ ] Tests cover each impacted area with the right priority.
- [ ] Score math is the weighted average of the five factors.
- [ ] No internal code details leaked.
- [ ] The existing PR description is preserved above the marker.