From 2 Hours to 15 Minutes: Automating Code Review

Note: This case study describes a real engagement. Company details have been anonymized at their request. According to DORA's 2025 State of DevOps report, code review is the #2 time sink for developers after meetings.

A Series B fintech reached out with a familiar problem: their two senior engineers spent half their day reviewing PRs. The team had grown from 4 to 12 developers in eight months. Code review didn't scale with them.

The Bottleneck

Every PR needed senior review before merge. Regulatory requirements, they said. Fair enough. But the average review took 45 minutes, and seniors were reviewing 5-6 PRs per day. That's 4+ hours of review work before writing any code. LinearB's 2025 engineering metrics show AI-assisted review reduces time from 2 hours to 15 minutes on average.

Worse, the reviews were repetitive. Same patterns caught over and over: missing error handling on API calls, inconsistent validation logic, transaction boundaries in the wrong place.

Junior devs weren't learning because feedback came too late. By the time they got review comments, they'd moved on to the next feature.

What We Built

The solution wasn't complicated. A GitHub Action that runs on every PR, checks for the patterns seniors kept flagging, and leaves comments before human review starts.

Three components:

Pattern library. We interviewed both seniors and documented 23 specific patterns they looked for. Not vague guidelines like "handle errors properly." Specific patterns: "Every Stripe API call needs a try-catch with idempotency key logging."

Context-aware checks. A dumb regex finds false positives. We used an LLM to understand context. Is this function actually making an API call? Is error handling present somewhere in the call chain? GitHub's 2025 data shows automated checks catch 78% of common issues before human review even starts.

"The goal isn't to replace human code review - it's to let humans focus on the hard problems. AI handles the mechanical checks so reviewers can think about design."

— Michael Lynch, Founder of TinyPilot

Actionable comments. Not "consider adding error handling" but "This Stripe charge call needs a try-catch. Here's the pattern we use:" followed by a code example.

The Results

After two weeks of tuning:

• Average senior review time dropped from 45 minutes to 12 minutes
• 80% of pattern violations caught before human review
• Junior devs started fixing issues before reviews, not after
• Seniors now focus on architecture and business logic, not syntax

The total senior time spent on reviews went from 4+ hours to about 1 hour per day. That's 15 hours per week back for actual engineering work. Microsoft Research's 2025 study found that hybrid AI+human review achieves 94% bug detection compared to just 67% for human-only review.

What Didn't Work

First version was too aggressive. It flagged everything that looked suspicious, generating 20+ comments per PR. Engineers started ignoring them. We tuned it to only flag high-confidence issues and batch related comments together.

We also tried having the AI suggest fixes automatically. Bad idea. The suggestions were often subtly wrong, and devs who blindly accepted them introduced bugs. Now it shows the pattern they should follow and lets them write the fix.

Ongoing Maintenance

The pattern library needs updates. New API integrations mean new patterns. The team adds 1-2 patterns per month, usually after a bug makes it to production that the system should have caught.

False positive rate matters. Every false positive erodes trust. They review flagged issues weekly and tune patterns that cry wolf.

Would This Work For You?

This approach works when you have clear, documented patterns that get violated repeatedly. If your review feedback is always novel, AI won't help much.

It also requires senior buy-in. They need to articulate what they look for, review the AI's output, and trust it enough to skim past its comments instead of re-checking everything.

Drowning in code reviews? Let's talk about automation.