You have never read the code

The long version

Nobody has ever read all of their code. We trust the layers below us because something or someone checks it, and because a few people kept reading it for us.

I’m calling that something intent verification: a change passes intent verification when a system, not a person reading the diff, can show that it does what you meant, inside the boundaries that matter. I believe it is what replaces readability as the property we design for, and that agentic coding is the next layer built on it.

The name is not new everywhere. Intent-based networking has used intent verification for years, and Mastercard announced “Verifiable Intent” with Google in March for recording what a user authorized an agent to do. Those share my goal, proving that an action matches what a person meant, but mine is about code, not networks or commerce. The closest prior art I’ve found on the code side is Shuvendu Lahiri’s work on intent formalization and the “intent gap.”

Verifiability over readability

Humans are slow at absorbing large amounts of text, so we optimized code to be easy to grok. Agents don’t read that way. They poke at code, run parts of it and see what happens, so what they need from a module is verifiability, meaning how easily they can watch it work.

Matt Pocock recently made this case, with a huge module full of complex performance tricks and 100% test coverage as his example: a talented engineer needs an hour or two to get comfortable in it, and an agent can lean on the tests and start right away. He keeps two reasons for human readability: a person can still jump in for spot checks on important pull requests, and the code should use the same domain language as the people describing it. (original post on X)

I think the direction is right. Design modules so they can be tested and verified on their own, and stop treating function size and cyclomatic complexity as the thing to optimize for agents. I’m less sure about the destination. Verifiable against what?

We have always worked at an abstraction

We never stopped reading code. We read it at the layer we work in, because our professional work has always been about the abstraction in front of us, not the ones underneath. Reading has never scaled past that layer, which is why every layer of the stack exists. Most Ruby developers read Ruby and never open the C that Ruby is built on, and few check the machine code a compiler emits. Each time a generation moved up a layer, it heard the same complaint: you don’t really understand what is running.

Some people do still read that layer. Compiler Explorer, a site for inspecting compiler output, was handling about three million compilations a week in 2022. A minority keeps reading so that the majority can trust.

Was the Ruby developer a worse engineer than the C developer? Was the C developer worse than the one writing assembly? I think that is the wrong question. They were solving different problems at different layers, and the ones who moved up could only do so because something below them was being checked. John Backus’s history of FORTRAN describes early skepticism that automatic programming could be efficient, and the fix was output close to hand-coded quality. Grace Hopper recalled that her compiler was accepted because it worked.

That trust was layered and partial, never absolute. Fuzzing found 325 bugs in mature compilers such as GCC and LLVM over about three years. The formally verified parts of CompCert had no wrong-code bugs, while its unverified front end had six.

xkcd 2347, titled “Dependency,” draws modern infrastructure as a tower of blocks resting on one small piece that a single person maintains. I read it less as a warning than as an observation. Software was never fully legible to any one person. We built layers so that it didn’t have to be.

Agentic coding is the next layer. What sits at its boundary is intent: what you asked for, in the words of your domain. Something has to stand where the compiler stood, and it has to check the result against that intent.

The design exists

I’d frame the last decade of DevOps practice as a long project in not reading code. Jez Humble and David Farley’s Continuous Delivery (2010) describes a deployment pipeline in which every commit that passes the first stage becomes a release candidate, and later stages exist to eliminate unfit candidates as early as possible.

The research that followed supports moving the gate. DORA’s 2019 State of DevOps report found that respondents with formal external change approval were 2.6 times more likely to be low performers, and saw no sign that it lowered change failure rates. That is survey data, so it shows association, not cause. It also shows where the human went. DORA’s recommendation is peer review during development, backed by automation. Nobody is arguing for removing people. The argument is for moving them closer to the work and letting the pipeline do the distant inspection.

Reading diffs was never mainly a defect filter anyway. At Microsoft, Bacchelli and Bird coded 570 review comments and found that 14% concerned defects, even though 44% of the 873 developers they surveyed named finding defects as their top motivation. Google’s review case study describes a process centered on understandability and norms.

Several practices from that era map onto agents, each with a known limit:

• Small batches and trunk-based development keep each change small enough to verify, though the practice itself lists eager code review as a prerequisite. Agents only produce small diffs if we ask them to.
• Contract tests, as in Robinson’s consumer-driven contracts, make a consumer’s expectations executable at a boundary, so you can verify a module without understanding its neighbors. They check the interactions consumers actually use, not business value.
• Feature flags and canaries (Hodgson; SRE Workbook) check a change against reality with a limited blast radius. The Workbook warns that small samples and short windows weaken the signal.
• Observability checks behavior you did not predict, although black-box monitoring only sees problems that are already happening.
• Reconciliation loops, as in Kubernetes controllers, compare declared desired state with observed state and act on the gap. A controller will enforce a wrong spec as faithfully as a right one.
• Error budgets make tolerated risk an explicit number and halt releases when it is spent. I’d expect a rate-based budget to fit poorly where a single wrong record matters, as in payments.

None of this is new, and the evidence says the machinery is not yet keeping up. DORA’s 2024 report estimated that a 25% increase in AI adoption was associated with a 7.2% drop in delivery stability and a 1.5% drop in throughput. Those are model-based associations, not proof of cause. The 2025 report calls AI an amplifier whose value depends on controls such as automated testing, version control and fast feedback. So the claim I can defend is that DevOps built the design. Whether your pipeline is strong enough to carry agent volume is a separate question. What changes is where the human sits: close to the work, with the pipeline as the first reviewer.

Agents add complexity, and it compounds

If nobody reads the code, the codebase still has to stay workable, and the main reader left is the agent. The early evidence says agents are not good at keeping it that way. He and colleagues compared 806 GitHub projects that adopted Cursor with 1,380 matched projects that didn’t. Adopters added lines three to five times faster in the first month, and that advantage faded within two months. Static analysis warnings rose 30% and code complexity 41%, and a panel analysis linked those increases to a later slowdown in velocity. That is observational, and adoption is inferred from the presence of Cursor configuration files, so it shows association, not cause.

Benchmarks point the same way. SlopCodeBench asks 15 coding agents to keep extending their own code across 36 problems and 196 checkpoints. Structural erosion rose in 77% of runs and verbosity in 75.5%, and next to 473 open-source Python repositories the agents’ code was about 2.3 times more verbose and 2.0 times more eroded. Telling the agents to write clean code lowered where they started by up to a third but, according to the authors, did not change how fast they degraded. A preprint called SWE-STEPS, which I have only read the abstract of, reports that testing agents one pull request at a time overstates their success by as much as 20 percentage points, because it ignores the mess earlier changes leave behind.

I would hold all of this loosely. These are early studies, several of them preprints, and the effect is not uniform. Borg and colleagues found that CodeScene’s code health score predicted whether five medium-sized open-weight models broke a program while refactoring it, but found no significant effect for Claude Sonnet 4.5 or for Claude working as an agent. A stronger model may need less help from the codebase. Even so, the direction is consistent: complexity piles up, and a pile of complexity makes the next change harder for whoever has to make it.

This is a problem for the thesis to solve, not an objection to it. If we stop reading code, nobody notices the pile growing. A prompt asking for quality is advice, and SlopCodeBench suggests advice doesn’t change the slope. What does is a check that can say no.

A deterministic evaluator around a non-deterministic author

The author varies from run to run, so the judge can’t. The idea is simple: let the agent write whatever it likes, and put a pipeline of deterministic checks between its change and the main branch. Same input, same verdict, no reading required. Each gate answers one question about the change, and one of them, a complexity budget, answers the problem above directly.

Here is a minimal version as a GitHub Actions workflow. I ran each of these commands against a small Python project to check the flags and exit codes, and the thresholds are placeholders to tune for your own codebase.

name: verify
on: pull_request

jobs:
  gates:
    runs-on: ubuntu-latest
    env:
      PYTHONPATH: src
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"
      - run: pip install ruff mypy pytest pytest-cov xenon import-linter mutmut

      # 1. Is it well-formed?
      - run: ruff check .
      - run: mypy src

      # 2. Does it behave? (coverage is a floor, not proof)
      - run: pytest --cov=shop --cov-fail-under=90

      # 3. Is the codebase still cheap for the next agent to change?
      - run: xenon --max-absolute B --max-modules A --max-average A src

      # 4. Does it respect the architecture?
      - run: lint-imports

      # 5. Would the tests notice if the code were wrong?
      - run: mutmut run
      - run: mutmut export-cicd-stats
      - run: python check_mutation.py 0.7

check_mutation.py is a dozen lines that read mutmut’s stats export and exit nonzero when the share of killed mutants falls below a threshold, because mutmut itself exits successfully even when mutants survive.

On my toy project, each gate caught the thing it exists to catch. A function with five parameters and nested branching scored a cyclomatic complexity of 11, rank C, and xenon failed the build under a B ceiling. A change that made the pricing module import from the orders module broke the import-linter contract and named the offending line. A test that called a function and asserted nothing still produced 100% coverage and passed the coverage gate, but all seven mutants of that function survived and the project’s mutation score fell to 59%. Adding two assertions brought it to 76%, and the survivors that remained were mostly boundary cases the tests never pinned down, which is a to-do list rather than a verdict.

The complexity gate is what changes the trajectory. An agent that adds a branchy function gets a red check and the failure output, and has to simplify before anyone is asked to look. The budget doesn’t care how the code reads to a person. It cares how much the next change has to hold in its head, whether that is a person or an agent. You can raise the ceiling or exempt a module on purpose, as Pocock’s performance-tuned module might deserve, but then it is a decision someone made, not drift. The failure output also doubles as a prompt: the agent reads the red check and fixes it, so the loop closes without a human.

What anchors the pipeline

A pipeline is only as good as what it checks against. Three things decide whether it deserves trust: the evaluator has to be deterministic, the spec has to say what you meant, and the tests have to be independent of the author.

A compiler is deterministic: the same input gives the same output, and decades of use have shaken out its bugs. An agent is not, which is why the verdict has to come from something that is. In an early study of ChatGPT, Ouyang and colleagues found that identical requests produced identical test results on only 47.56% of tasks in one benchmark (HumanEval) and 75.76% in another (CodeContests), even at temperature 0. Some of that can be engineered away: Thinking Machines made 1,000 temperature-0 completions identical on one open model, at roughly 1.6 times the run time. Agent runs still diverge early, though. Bjarnason and colleagues found single-run pass@1 swinging by 2.2 to 6.0 points.

The hard part is underspecification. A compiler bug is a deviation from a language standard, and conformance suites exist to check for it. An agent’s mistake is often a reasonable reading of an ambiguous prompt. Keles argues that this, not randomness, is the real problem, and Dijkstra made a version of the argument about natural-language programming decades ago.

Then there is circularity. If the same agent writes the code and the tests, a green pipeline shows that the two agree with each other. Konstantinou and colleagues found that LLM-generated test oracles mostly capture what the code actually does, not what it should do. In ImpossibleBench, where any pass is by construction a cheat, GPT-5 cheated on 54% of one task set and Claude Opus 4.1 on about half, and the Claude models did most of it by modifying the tests. The authors also found that the rate depends heavily on setup: a stricter prompt cut GPT-5’s rate on another set from about 92% to 1%. And coverage, the number in Pocock’s example, is a weak proxy. Inozemtseva and Holmes found it correlated only low to moderately with a test suite’s effectiveness once suite size was controlled.

What breaks the loop is not who writes the tests but whether something independent anchors them. In TiCoder, people approve tests instead of reading code, and the authors report accuracy gains, though from simulated feedback. Meta’s ACH generates tests aimed at injected faults, and Clover checks code, docstring and formal spec for consistency. Lahiri’s work adds the limit: code can be checked against a spec mechanically, but only the user can confirm that the spec matches what they meant. I haven’t found a study on whether a second model works as the independent check, so I treat that as untested.

Review also does a job that pipelines don’t. Bacchelli and Bird found that it produced knowledge transfer and shared awareness, which no green build gives you. Charity Majors argues that incidents in code nobody really understands get harder to resolve. Verification can approve a change, but debugging it at 3 a.m. still takes comprehension. That is Joel Spolsky’s law of leaky abstractions at work: someone has to be able to go down a layer.

That is why I keep saying intent rather than behavior. Verifying behavior asks whether the code does what the tests say. Verifying intent asks whether the tests say what you meant. I’m citing studies of the mechanism here, not measuring it on a real team, and I don’t have data on how often it bites in practice. I have seen the shape of it myself, though. On a healthcare data pipeline I worked on, the transformation and its tests shared the same misreading of what a field meant, so every check passed and the output was still wrong. A downstream user noticed that the results didn’t look right.

Where to start

If you want a team to work this way, these are the six things I would start with:

1. Write intent where a machine can check it. Use acceptance tests or executable specs, in the words your domain uses. The code and the people describing it share a vocabulary.
2. Give verification an independent anchor. Don’t let the same agent session write the code and decide what counts as passing. Anchor the check in something the author doesn’t control: human-approved examples, mutation testing, a consumer contract, a formal spec or a production signal. Coverage alone won’t do it.
3. Keep each change small and single-purpose. In one study of matched pull requests, agent PRs had a median of 48 added lines against 24 for human ones, and 39.9% were multi-purpose against 12.2%. DORA finds that small batches amplify AI’s benefits.
4. Make each module verifiable on its own. Put contracts at the boundary and keep tests fast enough to run on every change.
5. Save human review for what can’t be undone. Data migrations, anything that moves money and authorization boundaries are good candidates. It is still review. It just stops being the default gate for every line.
6. Verify in production, with a small blast radius. Flags, canaries and alerts tied to the intent, not only to uptime.

Fewer people, deeper

If this is right, team shape changes. Most Ruby shops don’t employ many C programmers. They rely on a small number of people who understand the layer beneath, and everyone else trusts the interface. There are precedents: Team Topologies describes small teams that own complicated lower layers for everyone else, and Google’s SRE function works in a similar way. Engineering organizations end up looking similar: a few people who can read, debug and own the lower layers, and a much larger share of the work done at the intent layer.

That is not a comfortable thing to say, because the specialists have to come from somewhere. Google’s SRE book says it hires a large share of SREs from standard software engineering backgrounds, so depth grows out of a broad base of people who did the work. Early evidence suggests that base is at risk. In a randomized study of 52 developers learning a new library, run by Anthropic (which makes Claude, a tool I use), those who used AI scored 50% on a comprehension quiz against 67% for those who didn’t, with the largest gap on debugging. Developers who delegated the work to the assistant scored lowest. It was a short study of a chat assistant, not of agents.

Lisanne Bainbridge described the same trap in 1983: automation erodes the skills people need when it fails, so the more advanced the system, the more crucial the human becomes. Depth matters when the checks miss something. The 2024 xz backdoor, as Qualys describes it, was found by an engineer who was investigating an SSH performance problem.

Reading code was never the goal. Knowing that the software does what we meant was the goal, and for a long time reading was the only tool we had.

So, a question for anyone running agents on a real codebase: what has your pipeline caught that a reviewer would have missed, and what has it let through?

Sources

• Matt Pocock, post on X, 4 Nov 2026.
• xkcd 2347, “Dependency”
• Humble and Farley, Continuous Delivery (2010), chapter 5 sample
• DORA, 2019 State of DevOps report; Streamlining change approval; Working in small batches; 2024 report announcement; 2025 report announcement
• Bacchelli and Bird, ICSE 2013; Sadowski et al., ICSE-SEIP 2018
• Watanabe et al., agent vs. human pull requests
• Practices: trunk-based development; Robinson, consumer-driven contracts; Hodgson, feature toggles; SRE Workbook, canarying; SRE book, monitoring; SRE book, embracing risk; Kubernetes controllers
• Compilers: Backus, History of FORTRAN; Hopper in Particular Passions; Yang et al., Csmith; CompCert; Godbolt, Compiler Explorer
• Determinism: Ouyang et al., arXiv 2308.02828; Thinking Machines, Defeating nondeterminism; Bjarnason et al., arXiv 2602.07150; Keles, LLMs could be, but shouldn’t be compilers; Dijkstra, EWD667
• Circularity and anchors: Konstantinou et al., arXiv 2410.21136; ImpossibleBench; Inozemtseva and Holmes, ICSE 2014; TiCoder; ACH; Clover; Lahiri, arXiv 2406.09757 and arXiv 2603.17150
• Review and comprehension: Majors, “You had one job”; Spolsky, The Law of Leaky Abstractions
• Team shape: Team Topologies; SRE book, introduction; Anthropic, AI assistance and coding skills; Bainbridge, Ironies of Automation; Qualys, xz backdoor
• Term: Forward Networks, intent-based networking; Mastercard, Verifiable Intent
• Agent-added complexity: He et al., Speed at the Cost of Quality; Orlanski et al., SlopCodeBench; Shastry et al., SWE-STEPS; Borg et al., Code for Machines, Not Just Humans
• Pipeline tools: ruff; mypy; pytest-cov; xenon; import-linter; mutmut