When AI makes code cheap, understanding becomes expensive

In my time working in technology I have seen two different kinds of software review culture. In one, pull requests were approved inside of thirty minutes and review was fairly light-touch focused on critical issues rather than an opinion on smaller improvements. In the other, review was genuinely thorough, (possibly too thorough) and pull requests could accumulate and sit for three days while somebody senior worked through what it actually did, what it was supposed to do, and what might go wrong on the unhappy path. This made that senior person incredibly valuable but also a constraint on delivery speed.

I have come round to actually thinking that both cultures made sense, given their respective situations. This stems from a combination of code standards, level of automation, interpersonal levels of trust and accountability, regulatory requirements, business demands and experience level of the teams involved.

Time is obviously a scarce resource, and all development teams end up optimising how much of it they spend on code review, usually without ever writing down an explicit targeted amount of time. In fact I'd go so far as to say many developers consider doing a code review of other peoples work to be a tax on their own productive time. When the yardstick for software development was how many {story points / tickets / features} have been delivered, subject to a constraint of sufficient levels of test coverage, you can see why code review is considered secondary.

Apart from ensuring a more correct codebase, there is of course an added benefit to having other people review the code, it reduces key person dependency (or the bus factor). The author of the code and the reviewers are now familiar with what the code is actually doing, the assumptions (implicit and explicit) and the design decisions.

Authorship & Knowledge used to come bundled

Here is the property of traditional development that I think is easy to miss precisely because it was free.

When somebody wrote code they produced several pieces of knowledge at once. They produced the implementation itself obviously. They also produced evidence that they had understood the problem, since you cannot hand-write something that works while holding none of the intent behind it (although mismatched requirements might suggest otherwise). They produced a known person who was accountable for the design decisions and quality. They also produced context around limitations of the implementation be it via test suites, documentation, code comments or just having a named person to ask why thigns are the way they are. There were also small updates to the developers own reputation within their team and organisation (for better or for worse).

Five things, and organisations usually only ever pay attention to the first one.

  1. the code itself
  2. demonstrated understanding of the problem
  3. accountability for decisions and quality
  4. context of the solution
  5. reputational increment

That does not mean developers understood their code perfectly, because they plainly did not, and anyone who has inherited a system knows how much of the understanding was thin or wrong to begin with. But authorship and understanding were coupled, and that coupling is incredibly important. If you wrote a pricing model you knew something about the model. If you designed a schema you knew something about the business rules underneath it.

AI code generation and agentic development unbundles that. A model can produce a perfectly plausible implementation without anyone in the loop possessing the corresponding mental model, and the prompt that carried whatever limited intent there was gets discarded straight afterwards. So the four things that used to arrive attached to the code now have to be considered separately and deliberately, and most teams have not started considering the longer term ramifications of this because they are too caught up being impressed with the speed of AI code writing.

This is not actually a completely new problem. People have shipped code they did not understand for as long as there have been libraries, code generators and Stack Overflow. What is new is the scale and cost: AI did not create the possibility of working without understanding, it made working without understanding cheap enough for it to become the default route to production.

The new bottleneck

Usage of AI to develop software speeds some elements up significantly but can also shift and possibly even diminish productivity in other parts of the lifecycle.

AI can take the creation of a piece of software from days to minutes. It does not do anything comparable to the time it takes to answer whether this is what we actually wanted, what it assumes, what happens when those assumptions fail, and how we would even know.

The asymmetry follows on its own. Generation scales almost without limit and human attention does not, so as generation gets cheaper, establishing confidence becomes a larger share of human productive time, until it is most of what is left. The scarce resource stops being the ability to write code and becomes the ability to establish that the code works correctly and is aligned with the original intent.

I might get around to economically modelling this out sometime but I'm going to make the assertion from experience that doing a proper code review on AI generated code takes more time than than on human written code.

The reviewer starts further from the intent, has to look at pull requests that can be substantially larger and more complex, have limited ability to shortcut their evaluation by relying on a colleagues reputation and their past experience of working with them, built up over months and years and have limited means of communicating with the author as to why decisions were made, the limitations and what they were really trying to achieve.

AI changes the economics of software development completely, but it's not a naive scaling as many assume. "AI can write code 20x faster than a person so we're now 20 times more efficient". This wrong assumption is being played out across many companies and to actually realise tangible increases in productivity requires looking at the entire lifecycle of software development, not just the code writing.

Won't AI just also review the code

If the author and the reviewer have made the same wrong assumption about what the code was for, more reading of requirements does not help, because they are not looking for the right thing. This is the real reason independent review works, namely that it is not another pair of eyes so much as another mental model, arrived at by a different route.

The same argument applies to generated tests, and it is worth stating carefully rather than dismissively. If a model writes the implementation and then writes the tests from the same reading of the requirement, the tests tend to confirm that reading rather than challenge it. The nearest analogy I have is checking a translation by translating it back with the same dictionary: if the dictionary has an error in it, the round trip is perfectly self-consistent and perfectly wrong, and the consistency is exactly what stops you noticing. Comprehensive test suites might give a false sense of security when the code generation process has misunderstood something fundamental in the first place.

I wouldn't say those tests are worthless, because they clearly catch things and prevent regressions. I would say however that evidence drawn from the same reasoning that produced the code confirms less than evidence drawn from somewhere else. What matters is independence of evidence and reasoning, not independence of execution, so four separate calls to the same model family with similar context are not four independent checks. (Even if the model families are different, it's highly likely that similar training data and feedback loops were used by the AI providors)

Ok fine so code reviews by a person are still required and in many regulated industries they may in fact always be required, simply for accountable decision reasons. But I hear you say, today that's maybe ~20% of a developers time so we could still expect a 3-5X increase in productivity, where developers spend more time understanding the code than necessarily writing it. That's getting more realistic but there's still one not insubstantial element to consider.

The reputation effect quietly disappears

The last piece is the one that gets overlooked entirely, and it is probably the biggest single loss.

A reviewer who has worked alongside an engineer for years already holds a great deal of intrinsic information about them: how careful they are, what kinds of mistakes they tend to make, whether their tests are any good, whether they know the domain. That history is what makes a light-touch review defensible. Reputation is really a form of compression, in that you do not re-derive everything because history lets you update a prior cheaply, and it was comfortably the cheapest source of trust the industry had. Either you had directly seen the quality of their work or could rely on weak signals from their credentials, projects they've contributed to or where they've worked. That's not to say you wouldn't challenge someone but you'd devote more time and attention to code from junior less experienced colleagues than to those who you were fairly certain knew what they were doing.

Agentic code development systems are evolving and continuously improving but today current systems mostly do not carry the persistent, accountable reputation structure that human teams have been using to price trust. A model's confident tone is not evidence, and I would say that AI developed code is trusted far less than code from senior human developers. This might change in time but the key takeaway is that AI written code reviews will take longer and need to be more thorough.

This is the now the very real bottleneck in creating enterprise grade software.

The economics of understanding

Understanding what the code is doing and being able to maintain software in production go hand in hand.

  • Option 1: Hope for the best, and accept reduced levels of understanding and thus quality and maintainability. This is the vibe coding approach and for personal projects or demos probably works fine. It ships in a weekend.

  • Option 2: Maintain the same rigour of quality, understanding and accountability but accept that the enormous speed up from AI writing the code will not translate into that same multiplier for end to end deployment.

  • Option 3: Don't use AI

I certainly don't think that the answer is to use less AI. The generation savings are significant and very real and we are only at the beginning of what's possible. The mistake is assuming that because code generation has become cheaper to produce, software has become proportionally cheaper to properly deploy. AI changes the cost structure rather than reducing it as much as hoped, and it moves the bottleneck to a place most delivery processes were never built to even look at. For decades writing code and understanding code were tightly coupled, so we got a great deal of understanding for free as a by-product of authorship, and that subsidy has now ended.

The cost of producing code is falling. The cost of knowing that the code is correct is not.

I'm betting on option 2, but accept that the development workflow for software also needs to adapt and change to be able to better leverage new capabilies in an AI first world. Towards that end I've put down an initial spec and approach for claim driven development (CDD)

Claim Driven Development

With the rise of generative code development workflows, we're really facing an age old problem under a different guise, namely that language is imprecise for defining intent and instead we must reach for something more mathematical to solve this.

The same motivating idea that lead to type systems, contracts, invariants and property-based testing is that a natural language sentence describing what a function should do cannot be checked, where as a property quantified over a domain can be. What is new is that natural language has quietly become the primary interface for expressing intent to the model writing the code, but at a scale where imprecision matters most given the scarcity of human attention and understanding. The code might still be technically correct with all tests passing but misaligned with what it was supposed to be doing or missing key caveats.

If the git diff is no longer reliable evidence of anyone's understanding, then reviewing the diff is no longer the right unit of work. The alternative is to make the intent itself (and its properties) the artefact: state what must be true about a function as a set of checkable claims, over a declared domain, before the implementation even exists.

The claim says what should be true, the implementation says how it is achieved, and the evidence says why we believe it. That gives a loop of claim, implementation, evidence, verification, rather than ticket, code, review, and the reviewed thing becomes a short list of statements in the language of the problem instead of a long complex diff in the language of the solution.

Of the five pieces of knowledge from code development discussed above, the first one is being handled by agentic coding extremely well, CDD aims to tackle the knock on effect of that on three of the other areas:

  1. demonstrated understanding of the problem
  2. accountability for decisions and quality
  3. context of the solution

The key premise for CDD is built on top of concepts like property based testing and TDD but with claims as the first class units of development as actual source code ultimately transitions to being more of a build artefact than a record of intent.

I'm hoping that tools conforming to this spec and implementing their own extensible grammers will allow for more robust and understood code, better developer experience and faster end-to-end deployment.