Build an AI code reviewer in 100 lines of Python
Your PR review process is broken and you know it. Here's the 100-line script I built at Stripe that catches real bugs before they hit production — not style nitpicks, actual logic errors that cost money.
Code review at most companies is theater. Senior engineer skims 400 lines of diff, leaves three comments about variable naming, approves it. Six days later you're on a bridge call at 2am because nobody caught the race condition.
I've been that senior engineer. I've also been the one paged at 2am.
The $340k bug that started this
In 2021, working on a 200-person eng team, we merged a payment retry handler that had a classic off-by-one error in the backoff logic. Retried too aggressively under load. Hammered our Stripe integration. The cascading failures cost us roughly $340k in lost transactions and three days of engineering time to untangle. It was in the diff. Nobody saw it. The PR had four approvals.
That's when I started building automated reviewers.
Here's the actual code
Stop reading tutorials that show you a chatbot wrapper. This does something specific: it reads your git diff, chunks it by file, sends each chunk to GPT-4o with a prompt engineered to find logic bugs, not style issues, and returns structured JSON you can pipe into your CI pipeline.
import subprocess, json, openai, sys
from typing import Generator
client = openai.OpenAI() # OPENAI_API_KEY in env
SYSTEM_PROMPT = """
You are a senior engineer reviewing a code diff.
Focus ONLY on:
- Logic errors and off-by-one bugs
- Race conditions and concurrency issues
- Unhandled error paths
- Security holes (injection, auth bypass, data exposure)
- Resource leaks
Ignore style, formatting, naming conventions.
Return JSON: {"issues": [{"severity": "high|medium|low", "line": "...", "description": "...", "fix": "..."}]}
"""
def get_diff(base: str = "main") -> str:
result = subprocess.run(
["git", "diff", base, "HEAD", "--unified=5"],
capture_output=True, text=True
)
return result.stdout
def chunk_diff(diff: str, max_chars: int = 6000) -> Generator[str, None, None]:
lines, current = [], 0
for line in diff.splitlines(keepends=True):
if current + len(line) > max_chars and lines:
yield "".join(lines)
lines, current = [], 0
lines.append(line)
current += len(line)
if lines:
yield "".join(lines)
def review_chunk(chunk: str) -> list[dict]:
response = client.chat.completions.create(
model="gpt-4o",
response_format={"type": "json_object"},
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": f"Review this diff:\n\n{chunk}"}
],
temperature=0.1 # lower = less hallucination
)
data = json.loads(response.choices[0].message.content)
return data.get("issues", [])
def main():
base = sys.argv[1] if len(sys.argv) > 1 else "main"
diff = get_diff(base)
if not diff.strip():
print("No diff found")
sys.exit(0)
all_issues = []
for chunk in chunk_diff(diff):
all_issues.extend(review_chunk(chunk))
high = [i for i in all_issues if i["severity"] == "high"]
for issue in all_issues:
severity = issue["severity"].upper()
print(f"[{severity}] {issue['description']}")
print(f" Fix: {issue['fix']}\n")
if high:
print(f"{len(high)} high-severity issues found")
sys.exit(1) # fails CI
if __name__ == "__main__":
main()Why temperature=0.1 matters
Don't skip that. Higher temperature means the model gets creative. You don't want creative. You want it to read the diff and find the bug, not invent one. I tested this at 0.7 and got false positives that made engineers distrust the whole tool. Trust is fragile. One bad false positive and your team ignores every future warning.
The prompt is the whole product
Most people write vague prompts and wonder why they get vague output. The system prompt above took me three weeks to tune. The key insight: telling the model what to ignore is as important as telling it what to find. Without "ignore style and formatting," you get 40 comments about semicolons and one missed SQL injection.
Wire it into GitHub Actions in 10 minutes
Drop this in .github/workflows/ai-review.yml and set OPENAI_API_KEY as a repo secret. Run it on pull_request events, diff against the base branch. When it exits 1, the PR can't merge. That's the whole integration. No vendor lock-in, no $400/month SaaS subscription, no sales call.
The tool won't replace a good engineer doing review. Nothing will. But it's awake at 3am when your engineer isn't, and it doesn't get tired on the 12th PR of the day.
What it won't catch
Don't pretend this is magic. It misses anything requiring business context. It doesn't know your invariants. It can't tell you that field X should never be null because of a contract you made with a partner in 2019. That's still on your engineers. Use this for the mechanical stuff. Free up human review time for the judgment calls.