TL;DR: When agents generate more than people can read, verification has to become a layered system, not one review at merge time.
- Shift objective checks (formatting, types, policy, many security and correctness issues) as close to authoring as possible.
- Keep changes small so every layer of review is faster and safer.
- Define acceptance criteria and tests up front so there's something concrete to verify against.
- Use an independent reviewer, and match review depth to the risk of the change.
When agents generate code faster than people can read it, review as a single human gate at merge time stops scaling. AI makes code cheaper to produce, but it doesn’t make software cheaper to own, and the scarce resource is still human judgment.1 The response is to treat verification as a layered system rather than one review step: deterministic checks, agent reasoning, and selective human judgment, each doing the part it’s best at. This page covers the layers and how to arrange them so judgment is spent only where it’s actually needed.
Shift objective checks earlier
Anything that can be checked by a fixed rule — formatting, type errors, dependency policy, many classes of security and correctness issues — should run as close to authoring as possible, not rediscovered at review. Every objective concern handled automatically frees human and agent attention for the problems that genuinely need reasoning. A violation caught in the editor costs seconds; the same violation caught at review costs a context switch and another round-trip.1
Keep changes small
Smaller changes are easier for both people and agents to reason about, easier to trace, and safer to roll back. Left to their defaults, agents tend to produce large, mixed diffs that combine refactoring, behavior changes, and formatting all at once, which buries the parts that matter. Keeping changes small and separating mechanical edits from behavioral ones makes every other layer of verification faster and more reliable.1
Define the behavior up front so it can be checked
Verification is far easier when behavior is expressed so it can be checked. Acceptance criteria defined before implementation, contract tests, and executable specifications give both people and agents something concrete to verify against, rather than inferring intent from the code alone. This matters more with agents, since implementation can be generated fast while the intent behind it is often underspecified. The question shifts from “does this code look reasonable?” to “does it satisfy the behavior we defined?”1
Start with how you will measure success. A statement of correct and incorrect behavior makes the eval close to obvious, whether you write the specification up front or build it through a question-and-answer loop with an agent. The first version will separate good from bad loosely, and tightening it is the iteration.
Find the cases worth a person’s time. Easy cases filter out automatically, and close calls need human judgment. Present each case as densely and intuitively as you can so a person can sort it quickly into "that's what I meant" or "that's not what I meant," then have an agent remove the groups of cases the person has ruled out. Starting from the complaints of the people who use the tool, such as "editing session titles is hard," gives you high-level objectives to turn into test cases.
Convert results into work. After an eval run, rank the cases that did not pass, turn each into a defect or feature request, and work through them. This resembles machine learning practice, such as hill climbing and clustering, more than a classic QA plan.
Check what the agent verified against. Agents sometimes build functionality to support the tests instead of testing the functionality you want, so write the requirements and test plan first. An agent's objective is behavior that plausibly matches the request. When an agent reports that all tests pass, ask what it checked against and who decided what correct means.
Turn what exploratory agents find into deterministic tests
Agents that drive a browser can click through an application faster than a person and surface problems that scripted checks missed. They are also slow and costly to run, so use them to find cases, then hand the cases to cheaper checks.
Give each exploratory agent a long-term motive, such as a merchandiser persona that wants to reason about gross margin. The agent moves through the running application and logs what it expected to see next and what surprised it. A merchandiser that sees revenue but no costs has found a gap. A person curates the surprises into a corpus of high-level scenarios, written in the language of your specification and observability data, and that corpus becomes an end-to-end holdout. Agents can write unit tests alongside it. Treat those tests as the agent’s ratchet within a task, which helps it backtrack less, and keep them separate from the question of whether the system is ready for production.
Make every finding reviewable. Exploratory agents need a visible session log and findings that another agent or a person can judge as interesting. Without them, a large volume of results turns into noise.
Budget for the cost. An eval set of 20 scenarios across seven areas with about 100 rubric items can take hours per run and use a large share of a token budget. Teams building this loop report spending anywhere from about three times to an order of magnitude more on validation than on generation. Treat that range as experience, not a benchmark.
Use independent or adversarial review
A review agent that shares the author agent’s assumptions tends to find the same things plausible and miss the same edge cases, so it agrees not because the output is correct but because its inputs matched the author’s. Introducing independence, a separate model, or at least a separate session with no access to the author’s reasoning, breaks that. The reviewer evaluates the result the way an unfamiliar senior engineer would, and catches problems the author agent was structurally unable to see.1 Building feedback and verification into delivery in GitLab Well Architected describes using representative evaluations, independent checks, and operating feedback to decide what can proceed.
Match review depth to risk
Not every change needs the full weight of review. Deciding how much verification a change gets, from a light pass to full scrutiny, should follow the risk of the change and the system it touches. A documentation change and a change to an authorization path don’t warrant the same depth, and tiering the review this way keeps cost and latency proportional to what’s at stake.1
Cap review iterations
More layers of review do not always produce a better review. One team built an initial AI review, then 11 review lenses, then a bar-raiser pass, then reran the full sequence two or three times after fixes. The justification was the cost of 11 human reviewers, a staffing level the team would never have funded, and a second pass should not cost as much as the first. Another team's reviews carried six layers plus an adversarial pass, which spent tokens to generate the code, again to review it, and again in continuous integration.
Expect four patterns:
- Without the original review context, the model finds something new each time. In one case, the same merge request run through an agent 10 times produced 10 different sets of findings.
- A fixed number of review iterations, after which the merge request is marked reviewed, avoids that loop.
- Long AI-written merge request comments cost tokens to write and more to read.
- When the final human reviewer runs their own AI tool over AI-reviewed code, the human pass repeats the automated one.
Pass earlier findings forward to each later review, and set a ceiling on reruns.
Ask why a check failed: Capability limit or context gap
When a check catches a bad output, the next question is why it failed, because the answer changes what you should do about it. Sometimes the agent is missing something you can give it, such as additional context or clearer boundaries on what it should do. Sometimes the agent cannot do the work yet, whatever context you add. Treat the two differently. Fold context and guardrail fixes back in right away so the same failure does not recur. Set capability gaps aside to revisit as models improve, and keep a human checkpoint there in the meantime, rather than building processes around a problem no process will solve.
Where practitioners disagree: The shape of the test suite
The classic test pyramid has many unit tests, fewer integration tests, and few end-to-end tests. Practitioners have not settled how it applies when agents write the code. Three distinct points of view have emerged:
- Design first. Effective unit tests depend on design, specifically separating one dimension from another so each can be tested alone and then together. Agents are weak at structure, so expect fewer and less impactful unit tests. Agents can redesign code for testability when prompted, but they don't by default. Tests coupled to structure, which once made refactoring costly, may now cost less, though this is unproven.
- Economics changed, physics didn't. Work in small units, because that is still how good software gets built. End-to-end tests remain more expensive than unit tests but cost much less than before, because agent effort is cheaper than human effort. The right shape may be closer to a cylinder than a cone.
- Keep the pyramid. Teams running revenue-critical systems keep the ideal pyramid and add a larger layer of manual acceptance and exploratory testing at the top.
One constraint applies to all three: An agent iterating toward a passing build should not wait on slow tests, and agents that write slow tests well but fast tests poorly produce a loop that stalls.
In practice: Verification as production infrastructure
One infrastructure company1 built its AI code review into its CI pipeline, so review runs as part of delivery. Instead of a single model summarizing a diff, it runs several specialized reviewers in parallel, for security, performance, code quality, and other concerns, and a coordinator consolidates their findings into one structured review: deduplicated, severity-ranked, and posted back to the change. Review depth is tiered by risk, small changes get a lighter pass while larger or security-sensitive ones trigger the full set, and noisy files like generated artifacts are filtered out before any model reasons about them. A human “break glass” override stays available for the cases that need it.
Two things make this trustworthy. Because it runs in the delivery path, it needs the operational discipline of any production system, retries, timeouts, telemetry, and a way to change model routing without touching the pipeline, so reliability becomes part of the review’s credibility. And the company is explicit about the limits: architectural judgment, cross-system impact, and subtle concurrency bugs remain weaker spots, which positions the system as an augmentation layer that raises the floor and reserves human reviewers for the reasoning only they can do.1
Try this. Take one kind of comment your reviewers keep making by hand, and move it into an automated check that runs before review. Each recurring judgment you can turn into a deterministic check is attention returned to the problems that actually need a person. The Code Style Enforcer agent in the Agentic AI Cookbook shows one way to move a recurring style comment into a check on every merge request.
Related
- Make the codebase and its context agent-ready comes first: review agents reason well only when they can retrieve the right context.
- Match the method to the task covers the deterministic-versus-agent choice this page applies to verification.
- Earn autonomy with evidence depends on verification, since you can widen an agent’s autonomy only as far as you can catch its mistakes.


