Require Evidence for AI-Generated Code

Revised 28 August 2026 at 20:30
Published 04 May 2026 at 05:32
~ 6 min read

#ai #automation #software engineering #testing

Coding agents can produce changes faster than many teams can review them. Require each change to carry enough evidence to show that it meets the stated behaviour, passes the relevant checks and can be operated safely.

The useful measure is not generated code or completed tasks. It is whether changes reach production with acceptable review time, failure rate and maintenance cost.

Verification is the constraint

Recent coding agents can inspect a repository, edit several files, run commands and revise a change after failures. That makes them more useful than autocomplete, but it also changes the review workload.

Most delivery processes still assume implementation is the slow part. Requirements can remain vague because a developer clarifies them while working. Tests and security checks often run after implementation. Review capacity is planned around human coding speed.

When an agent produces several plausible implementations before a reviewer has assessed the first, verification becomes the constraint. Faster generation can increase queue depth and rework without improving delivery.

Research points to this gap. Google’s 2025 DORA summary describes AI as an amplifier of the engineering system it enters. Faros reported higher task and pull-request throughput alongside longer review times in its AI Productivity Paradox report. These studies do not establish one result for every team, but they are enough to reject generated volume as a quality measure.

Abstract illustration of AI-generated code throughput narrowing into a verification bottleneck

Define the result before generating code

An agent should not invent the goal while also inventing the implementation. Before it edits the repository, record:

the expected user or system behaviour
examples of accepted and rejected behaviour
architectural and data constraints
the risk class
required security, performance and observability checks
the release and rollback approach

This can be a short issue template. The purpose is to give both the agent and reviewer a testable contract.

Acceptance criteria should describe behaviour rather than implementation. “Return 403 when an account lacks the billing role” is useful. “Add a permissions helper” assumes a design before the repository has been inspected.

Attach evidence to the change

The pull request should state what the agent changed, which checks it ran and what remains uncertain. A small structured record makes that information easier to review:

change:
    intent: "Add account-level API key rotation"
    risk_class: "high"
    ai_assisted: true
    scope:
        files_changed:
            - "app/security/api_keys.ts"
            - "app/routes/account/api_keys.ts"
            - "tests/api_keys.test.ts"
        services_touched:
            - "account-service"
            - "audit-log-service"
    evidence:
        acceptance_criteria_checked: true
        tests_added: true
        commands_run:
            - "npm run lint"
            - "npm test -- api_keys"
            - "npm run typecheck"
        security_review_required: true
        rollback_path: "feature flag: api_key_rotation_enabled"
    open_questions:
        - "Should old keys remain valid for 24 hours or 7 days?"

The schema is less important than its accuracy. Do not let an agent claim a command ran unless the captured output or CI result confirms it. Record skipped checks and the reason. Reviewers should not have to infer the test scope from a green badge.

Abstract illustration of an AI-generated change carrying a connected evidence bundle for review

Apply review according to risk

Line count is a poor proxy for risk. A small authentication change can require more scrutiny than a generated form with hundreds of lines.

Risk class	Example	Minimum gate
Low	Documentation, copy or bounded internal tooling	Automated checks and lightweight human approval
Medium	Feature work in a well-tested, owned area	Human review of behaviour, tests, edge cases and maintainability
High	Authentication, billing, customer data, migrations or APIs	Senior review, security analysis, staged rollout and rollback path
Critical	Irreversible data operations or regulated, safety-critical work	Human-led implementation or close pair review, with no auto-merge

Risk classes should reflect the repository and operating environment. The aim is to spend scarce reviewer attention on the changes where a mistake is expensive.

Keep generated changes small. Separate mechanical refactors from behaviour changes and reject unrelated edits. A narrow diff is easier to compare with the stated intent and cheaper to abandon when the proposed design is wrong.

Make checks part of the generation loop

Run deterministic checks throughout the task and again before review:

Convert acceptance criteria into executable examples where practical.
Run formatting, linting, type checks and relevant tests after each bounded edit.
Add dependency, secret and static security scans where the risk warrants them.
Use an independent review pass that did not author the change.
Release risky work behind a flag or canary and monitor the expected signals.
Link incidents and corrective changes back to the original pull request.

Security controls should assume that some generated code will be wrong. Least privilege, runtime isolation, rate limits, audit logs and a tested rollback path reduce the effect of a mistake without relying on the model to avoid every flaw.

The UK National Cyber Security Centre makes a similar case in its guidance on AI-assisted development: human review alone will not absorb unlimited generated output, so platforms need secure defaults and deterministic controls.

Abstract illustration of AI-generated code passing through deterministic security and policy guardrails

Measure the delivery system

A useful dashboard connects generation to review and production outcomes. Start with a small set of measures:

cycle time and review latency by risk class
pull-request size and review queue depth
relevant tests run and checks skipped
change failure and rollback rates
escaped defects and time to first corrective change
ownership coverage for AI-assisted changes

Track model and tool versions only when that provenance helps investigate a result. Avoid turning the dashboard into a count of prompts, tokens or generated lines.

Public coding benchmarks can show broad capability, but they do not test a team’s architecture, historical failures or domain rules. Build internal evaluations from real bugs, migrations, security findings and code-review examples. Those tasks answer the operational question: can this workflow produce an acceptable change in this repository?

Keep ownership with the team

A passing change can still make a codebase harder to maintain. Review dependency changes, duplicated abstractions, coupling and the amount of corrective work that follows. More importantly, make sure a current engineer can explain and operate the changed behaviour.

Humans should retain responsibility for intent, architecture, risk acceptance and production readiness. Agents can implement, test and analyse work within those boundaries. That division is useful only when the evidence makes the boundary visible.

Start with low-risk, well-tested tasks. Require an evidence record, measure the review and production results, then increase autonomy only when the data shows that the team can absorb more generated work safely.