# AI-Driven Development: The Practices That Have to Change

*Published 2026-07-25 · Dmitry Shirokov · shirokoff.ca/blog/ai-driven-development-practice*

A team of six merged more code in a month than the previous quarter. Cycle time collapsed. It went in a board update. Then incidents crept up, and on-call started finding failures in code nobody could explain — because nobody had written it and, it turned out, nobody had really read it. The agents hadn't written bad code. **The team multiplied its ability to produce changes and left its ability to verify them exactly where it was.**

## The research, before the opinions

**METR's randomized trial** (mid-2025): experienced open-source developers were ~**19% slower** on real tasks in codebases they knew well when allowed to use AI tools — and estimated afterwards they'd been ~**20% faster**. A ~39-point gap, in the direction that flatters the tool. **DORA's 2025 report**: near-universal adoption, and AI acting as an **amplifier** of existing strengths and dysfunctions.

Operating assumption: **your team's confidence that this is going well is not evidence, and whatever your delivery system was good and bad at, it is now more so.** Not an argument against AI-driven development — the teams doing it well are genuinely faster — an argument for measuring outcomes, and fixing practice before scaling tooling.

## Review: the practice that changes most

**Cap reviewable change size and enforce it** — review quality falls off a cliff past a few hundred lines, and that predates AI. If the agent can generate more than you can review, review is the constraint. **Review against intent, not style**: does this do what the spec said? Is it the smallest change that does? What did it touch unnecessarily — and **what did it delete**? That last question catches a nasty class of behaviour: a failing test "fixed" by weakening its assertion. **The author must be able to explain every line** — write it into the working agreement; the alternative is code with no human who understands it. **Make provenance explicit** in the PR, not to shame anyone but to route review attention.

A PR template that works: what & why (link the spec), how it was built (hand-written / assisted / agent-authored from spec), and **verification — what *I* did, not what the agent claims**: I ran it and observed X; the tests would **fail without this change**; I checked what was deleted or weakened; dependencies added (none is the expected answer); I can explain every line. Plus blast radius: what breaks if this is wrong, and how would we notice?

## Tests: now the specification, and now co-authored

When an agent optimizes toward "tests pass," the suite becomes the spec the system is fitted to — a real upgrade in leverage with a trap: **if the same agent writes code and tests in one pass, the tests encode the same misunderstanding.** They'll pass and prove nothing.

Controls, by value: **write acceptance tests from the spec before implementation** (by a human, or an agent that never sees the implementation) — the single most effective one, because it makes the test an independent statement of intent; **insist tests fail without the change**; **prefer properties and contracts at boundaries**, which are harder to satisfy accidentally. And coverage is not the metric — an agent can produce 95% coverage of code nobody needs.

## Context as a first-class artifact

Put engineering standards where the agent reads them: `AGENTS.md` at the repo root (read natively by most agents now) with build and test commands, conventions, architectural boundaries, off-limits directories. Three rules: **keep it short and true** (a stale instruction is worse than a missing one — the agent follows it confidently), **review it like code**, and **prefer executable rules over described ones** — a linter, formatter, scaffold or failing test enforces a convention every time; a paragraph is advice.

## Ownership, metrics, and the signature

**Ownership**: who is on call for this and can they debug it under pressure? "The team owns it" means nobody does. **Metrics**: DORA's four still tell the truth, but throughput (deployment frequency, lead time) improves almost automatically and flatters you. Watch the stability pair — **change failure rate** and **time to restore**. The signature of scaling generation without verification: *throughput up, change failure rate creeping, time-to-restore worsening* because the debugger didn't write the code. Add **review latency** (a direct read on the new bottleneck) and **rework rate** (changes to code merged in the last 30 days). **Supply chain**: agents add dependencies with a cheerfulness no human reviewer shows — every one is a supply-chain decision.

## The people question

Juniors learn by doing the work agents now do. If their job becomes reviewing generated code, they're exercising judgment they haven't built, by a path that's been removed. No tidy answer; what I've seen work: reserve some work as deliberately hand-written, pair on the **review** of agent output (reading code critically is teachable and now central), and rotate people onto on-call for systems they didn't write — where the gap between "I approved this" and "I understand this" gets concrete fast. Senior-side: METR suggests individual judgment ("this is helping me") is unreliable, so look at outcomes over weeks, not how the afternoon felt.

## Three failure modes that look like success first

**Rubber-stamp review** — approval latency down, approval rate near 100%, comments-per-review near zero; those three together are the clearest early warning, and it's already in your tooling. **Test co-authorship** — coverage climbing alongside escaped defects; the tell is a suite that has never caught a regression in agent-authored code. **Velocity theatre** — the board update reports merged PRs, the incident review reports something else, and nobody joins them. *Report throughput and stability on the same slide, always.*

## Where I'd start

1. Write `AGENTS.md` (an afternoon). 2. Cap PR size; make the task the unit of work. 3. Adopt the PR template, especially "would fail without this change" and "what did this delete." 4. Put change failure rate and review latency next to throughput on the dashboard leadership already reads. 5. Write acceptance tests from the spec before implementation. 6. **Then** expand tooling — scaling tools ahead of practice scales the dysfunction too.

*Related: [Spec-driven development](https://shirokoff.ca/blog/spec-driven-development) · [DORA metrics for data and AI](https://shirokoff.ca/blog/dora-metrics-data-ai) · [Sandboxing an AI agent](https://shirokoff.ca/blog/agent-sandboxing-tool-isolation) · [The agent harness](https://shirokoff.ca/blog/agent-harness-design) · [Estimating AI projects](https://shirokoff.ca/blog/estimating-ai-projects) · [AI governance frameworks](https://shirokoff.ca/blog/ai-governance-frameworks)*