Your AI 10x is real, and so are the studies that say it isn't

The perception trap is real, and it is measured

The most uncomfortable finding in the METR trial is not the 19 percent. It is what the developers believed. Before the tasks, they forecast that AI would make them 24 percent faster. After finishing, having actually been slower, they still believed AI had sped them up by about 20 percent.

Sit with that. People who write software for a living, measured on their own machines, were wrong about their own speed by roughly forty points, and confidently so. This is the productivity paradox in one data point: "it feels faster" is not evidence. If your case for an AI rollout rests on how fast the team says it feels, you do not have a case. You have a vibe, and the vibe is measurably unreliable.

Where the big gains are real, and why they do not generalize

This is the part the skeptics skip. AI is genuinely, dramatically faster in the right shape of work. The Nielsen Norman Group reported a 126 percent speedup on a coding task, more than double. But look at the task: build an HTTP server from scratch, a scoped, self-contained, greenfield problem.

That is the tell. AI shines on bounded, greenfield, low-context work, and that is precisely the workflow least like a developer's daily life inside a large, mature codebase with history, constraints, and a hundred ways to break something three modules away. The 126 percent is real. It is also real that almost none of your team's actual work looks like that task. The gains are not fake. They are task-shaped, and most work is the wrong shape unless you reshape it.

The bottleneck was never typing

Here is why faster typing does not move the median number. DX found a net-zero dynamic: the time AI saves on writing code gets eaten back by reviewing, validating, and remediating what the AI produced. And coding is only about 16 percent of a developer's time to begin with. Make 16 percent of the job faster, then spend the savings cleaning up, and the system barely moves.

Uplevel saw the harder version of this. Across roughly 800 developers, it found no significant change in throughput after adopting Copilot, and a 41 percent increase in bugs in pull requests. If the real bottleneck is review, queues, and decisions, then accelerating the one upstream step that was never the constraint just pushes more work, and more defects, into the step that was.

What AI actually changes is the shape of the work

The most useful result is the quietest. An MIT Sloan analysis of 187,000 GitHub developers around Copilot's launch found that coding activity rose 12.4 percent while project-management activity fell 24.9 percent. AI did not simply make people faster. It moved their time back toward the core of the craft and away from coordination.

That is the actual lever, and it only pays off if you redesign around it. If you take a developer whose time just shifted toward real engineering and drop them back into the same review queues, the same meetings, and the same ship process, you have changed the inputs and kept the bottleneck. You get DX's 8 percent. If you redesign the workflow so the freed attention lands where the constraint actually is, you get something that looks like the gains the optimists promise.

Same lesson as cloud

This is where it connects to the rest of what we have been writing. The median developer in these studies is doing to AI exactly what most companies did to cloud: lift-and-shift. Keep the process, keep the habits, bolt the new thing on, and book a worse version of what you had. The developers and teams getting the real return are the ones who rebuilt the workflow around the tool, smaller scoped units where AI is strong, human attention moved to the review and decisions where the bottleneck lives.

We covered the business version of this in AI is the new lift-and-shift and the playbook for steering it in how to keep AI from becoming lift-and-shift. This is the same lesson at the level of one keyboard: the win was never the tool. It was the redesign.

What I am not claiming

I am being honest about the limits, because the whole point is that the feeling is not the proof. METR is 16 expert developers on their own mature repositories, a small and specific sample that does not generalize to every team. The 126 percent gain is one scoped greenfield task, not whole-codebase productivity. And my own 10x is one operator's lived experience, not a study. I am offering it as a direction, not a multiplier you can put in a budget. The defensible claim is not a number. It is that the operating model, not the tool, decides whether AI helps you or quietly costs you.

One-sentence takeaway

The studies that say AI does not make developers faster and your own sense that it makes you much faster are both right, and the difference between them is whether the workflow was redesigned around the tool or the tool was bolted onto an unchanged one.