Developer reviewing AI code verification results on a split screen beside flagged pull requests

Your coding agent writes a 2,000-line pull request in four minutes. Your team needs a week to review it. That gap is where the cost now lives. Faros analyzed two years of telemetry from 22,000 developers across 4,000 teams and found 54% more bugs per developer, roughly five times longer median PR review times, and more than three times as many incidents per PR in periods of higher AI adoption. Throughput went up. So did rework, review load, and production incidents.

The constraint has moved from writing code to verifying it, and human review alone cannot absorb the volume. Below, we look at what practical AI code verification looks like in 2026, including agentic methodologies like Superpowers and continuous review frameworks like Wes McKinney’s RoboRev, plus what each one actually catches.

Throughput Is Up, So Why Are Bugs and Incidents Up Faster?

Faros was careful about one thing, and most people repeating the headline numbers skip it: these are correlations, not proof that AI caused every bug or outage. Teams adopting agents fast tend to be different teams, working on different codebases, under different pressure. The telemetry shows what happened alongside adoption. It does not prove the mechanism.

That caveat does not make the bill disappear. A follow-up Q3 report found some of the early effects easing while verification, remediation, and operational costs kept piling up downstream. Generation got cheap. Checking did not.

So the cost moved rather than vanished. It landed on reviewers, on-call engineers, and support, none of whom had that capacity budgeted. Until AI code verification scales at the same rate as generation, every speed gain upstream becomes someone else’s workload later.

Chart showing 54% bug increase across 22,000 developers, highlighting gaps in AI code verification

What Automated Software Verification Actually Means Here

Automated software verification is a layered set of machine-run checks that establish whether code does what it claims before anyone ships it. Type checks, property-based tests, fuzzing, static analysis, security scanning, contract tests, and a second model reviewing the first model’s output. None of it promises bug-free software. The point is to catch more problems earlier, at a volume no reviewer can match.

This is different from code review as most teams practise it. Traditional review assumes a human reads the diff, understands the intent, and spots what is wrong. That assumption held when a developer wrote 200 lines a day.

Justified trust vs. assumed trust in generated code

Assumed trust is approving a pull request because the tests are green and the agent sounded confident in its summary. Justified trust is approving it because specific checks ran against specific risks, and you can name which ones.

The difference matters most where it is expensive. An agent that invents a plausible-looking data transform will pass a smoke test and corrupt a production batch record three weeks later. Verification has to target the failure modes you actually care about, not just compile success.

Why human review stops scaling at agent speed

As the source puts it plainly, “One cannot expect a small team to meaningfully review enormous AI-generated diffs.” Attention does not scale with diff size. Past a few hundred lines, reviewers skim, approve, and hope.

The fix is to point the same tooling at the checking that you pointed at the writing. Agents, harnesses, and skills that generate code at speed can also generate test suites, adversarial cases, and review passes at that speed. Humans then review the findings, not the raw diff.

Superpowers vs. RoboRev: Two Different Bets on Verification

Two approaches have gained traction, and they attack the problem from opposite ends. One tries to stop bad code being written. The other assumes bad code will be written and catches it immediately after.

Superpowers: discipline before the first line of code

Superpowers is an agentic development methodology and collection of skills that plugs into coding agents and supported harnesses. Instead of letting the agent sprint straight to implementation, it forces the behaviour you would expect from a careful engineer: clarify what is actually being built, plan it, write tests first, debug properly, and hand the work to separate reviewer agents.

The trade-off is real. Running the full methodology slows delivery, and on small or routine tasks that overhead is hard to justify. It also does not cover the broader correctness and security checks a system needs. No plugin makes software bug-free or vulnerability-free, and treating one as a quality gate is how teams end up surprised in production.

RoboRev: a second model reviewing the first model’s commits

RoboRev comes from Wes McKinney, the developer behind pandas. It runs a local daemon that reviews each commit automatically using a configured coding agent, then surfaces findings you feed back into your current LLM and harness. Continuous AI code review inside the development loop, not a queue item waiting for a human.

The logic mirrors peer review. A separate reviewer, ideally running a different model, catches mistakes and architectural problems the authoring agent was blind to. It is another perspective, not a guarantee of an unbiased or correct one. Models share training data and share failure modes, so a second opinion narrows the gap without closing it.

Side-by-side diagram comparing Superpowers skill workflows and RoboRev review passes for AI code verification

Where a Second AI Reviewer Helps and Where It Falls Short

A separate reviewing agent does real work. Wes McKinney built RoboRev on exactly the logic that makes peer review valuable: a reader who did not write the code carries none of the author’s assumptions. Run it on a different model and it catches naming inconsistencies, missed edge cases, and architectural drift that the authoring agent was blind to by construction.

What it does not give you is an unbiased or correct review. It gives you a second opinion from a system with its own failure modes. Treat its output as findings to triage, not as a sign-off.

Correlated models, correlated blind spots

Two frontier models trained on overlapping public code tend to make overlapping mistakes. They will both be confident about the same deprecated API pattern, both miss the same race condition, both accept the same plausible-looking authentication shortcut. Model diversity reduces this, it does not remove it. No plugin makes software bug-free or vulnerability-free.

That is why the deterministic layer stays non-negotiable. Type checks and a real test suite either pass or fail. Static analysis and dependency scanning catch classes of vulnerability no reviewer looks for on a Tuesday afternoon. Production observability tells you what actually broke, which is the only verification signal that is not an opinion.

The economics back this up. Faros found that even as some of the early adoption effects eased, downstream verification and remediation costs kept accumulating. The spend moved, it did not disappear. If your AI code verification strategy is one more agent reading diffs, you have added a useful voice to the conversation and left the expensive part, finding out what is genuinely broken, unsolved.

Building a Verification Layer Your Team Can Actually Run

Start by measuring what you have before you add agents. Bugs per developer, median PR review time, incidents per PR: three numbers, pulled from your existing issue tracker and CI logs. Without a baseline you will be arguing about whether adoption helped, using anecdotes. Faros only produced useful findings because it had two years of telemetry to compare against.

Then cap the size of what an agent is allowed to produce in one change. Large diffs are the root cause, not the symptom. A hard limit forces decomposition, and small changes are reviewable by both humans and machines.

The four checks to automate first

  • Type checking and static analysis: deterministic, fast, catches a whole class of agent errors without judgement calls.
  • Tests written as part of the agent’s definition of done: no tests, no merge. The agent writes them, the pipeline runs them.
  • Security and dependency scanning: agents pull in packages confidently and wrongly.
  • An independent reviewer agent on a different model: findings only, never a merge gate.

Gate merges on the deterministic checks. Agent opinion goes in a comment thread for a human to triage. Mixing the two means a probabilistic system holds the keys to production, which is exactly the problem you were solving.

Metrics that prove the verification layer is paying off

Track the same three baseline numbers monthly, plus one more: what percentage of defects were caught before merge rather than in production. That ratio is the whole case.

Compute and tooling for verification cost very little next to an incident response weekend, a rework sprint, or a two-week review backlog. If your pre-merge catch rate climbs while review time holds steady, the layer is working.

Operations lead mapping an AI code verification workflow on a whiteboard with sticky notes

Ready to find AI opportunities in your business?
Book a Free AI Opportunity Audit. It is a 30-minute call where we map the highest-value automations in your operation.

The 2026 Engineering Org Spends More on Checking Than Writing

Writing code is close to free now. That means it no longer differentiates anyone. What separates teams is how fast and how credibly they can establish that the code is correct, and that capability has to be built, maintained, and owned like any other product.

Most organisations still treat verification as a plugin. A linter here, a scanner in CI, a reviewing agent someone configured on a Friday. That is not a verification layer, that is a collection of habits. The teams pulling ahead assign ownership, track whether checks actually catch defects, and retire the ones that only generate noise.

Manufacturing leaders already know the argument. You do not inspect quality in at the end of the line. Final inspection catches scrap after you have paid to make it, and it tells you nothing about why. In-process control, poka-yoke, statistical checks at each station: these work because they stop defects from travelling downstream. Automated software verification is the same principle applied to code, and the economics are identical. A defect caught by a property test costs a fraction of one found in production.

The source article puts the goal plainly: verification that builds justified trust and confidence. Not guarantees. Justified confidence, backed by evidence you can point at.

In the next 30 days, review four things. Who owns verification tooling by name, and what percentage of their time it gets. What your checks actually caught last quarter versus what reached users. Whether agent-generated changes bypass any gate a human change would hit. And whether your remediation cost is visible in a report anyone reads, or buried in the sprint as unlabelled rework.

Source: mihaisplace.com

Leave a Reply