Feedback Is the New Bottleneck

I recently wrote about how code review is dying as a gatekeeping mechanism. Pull requests take seconds to create and hours to evaluate. Since writing that post, the problem has become personal.

I used to work with AI interactively, prompting back and forth on a single task. Now I delegate. Whole chunks of work run concurrently: an agent drafts a blog post while another processes a meeting transcript while another builds a feature. I come back to a queue of completed work that needs reviewing. Each agent runs with the same energy on its thirty-seventh task as it did on its first. I do not. I am one person trying to review all of it, and I am the bottleneck.

I built Lockbox as a first step, solving the security side of this problem by tracking what agents have been exposed to and blocking dangerous follow-up actions.¹ But security is only half the story. What is the right interface for reviewing and unblocking AI?

Researchers found the same pattern at scale. Engineers who reported being 20% faster with AI were part of teams delivering 20% slower overall.² A separate study of over ten thousand developers found that teams with high AI adoption completed 21% more tasks and merged 98% more pull requests, but PR review time increased 91%, with no measurable improvement in delivery velocity.³ This is the theory of constraints in action: speed up one stage and the bottleneck moves downstream. When code arrives faster, it pushes more work into review, testing, deployment, and requirements clarification. The queue grows. Nobody is reviewing any faster.

The Exhaustion Gap

By mid-afternoon, the human reviewing and validating is running out of steam.

With human colleagues, natural rhythms emerge. You both get tired, take breaks, and have energy that ebbs and flows in patterns you understand. With AI, the human is the only one who needs rest while the machine keeps generating work that needs evaluating: permission fatigue, review fatigue, the endless “just need a human to press approve” requests. Cory Doctorow calls this the reverse centaur: humans whose purpose is to support the machine’s needs.⁴

The reviewers who care most about quality will be the first to burn out, because they are the ones who read everything instead of skimming. Either you make human feedback unnecessary, or you make it instant.

Make Feedback Unnecessary

StrongDM’s Software Factory creates synthetic replicas of services like Okta, Jira, and Slack, letting agents validate against realistic environments without hitting real APIs.⁵ The output is a pass/fail report, not a code change for a human to inspect. The agent tests its own work against a realistic simulation and only escalates failures.

I use the same principle with slide decks. My AI generates slides, then reviews each one against design criteria before showing it to me. About one in ten gets regenerated because the agent caught the problem first. I never see the flawed version.

I built a synthetic audience of CTO personas, some generated from transcripts of real conversations with actual CTOs, and run draft content past them before publishing. They catch structural problems, flag assumptions that do not land, and test whether the argument holds for different experience levels. A proposed email tested against a customer persona flags tone issues before you send it.

The pattern is the same whether the domain is code or content: build an intermediate check that runs at machine speed, so the human only reviews what survives.

Make Feedback Instant

Zack Akil from Google put it well at a recent LLM London meetup: an agent should produce output that is instantly verifiable by the person reviewing it. He gave the example of a medical explainer generated by AI. You would never put that directly in front of patients, because the risk of a hallucinated detail is too high. But show it to a doctor first and the calculus changes completely. The doctor can scan it in seconds, spot anything wrong, approve it, and the patient gets a better explanation than most clinics produce manually. The AI is not safe enough to replace the expert, but it is good enough to draft something the expert can verify quickly.

Imagine if Perplexity showed you a spinning circle for sixty seconds, then an answer. Instead, it shows every search step, every source, every thread of reasoning as it works.⁶ You can check the logic before the final answer arrives.

For code, diffs are not the answer. A 500-line diff generated by an AI agent is reviewable, but it takes an age and you cannot maintain a good level of scrutiny all day — which is how we ended up with 91% longer review times. Cursor took a different approach: their agents now run the software they build and send you a video of it working.⁷ You watch the app instead of reading the diff. A two-minute demo tells you more than twenty minutes spent staring at green and red lines. Devin went further, automatically fixing review comments so the loop between feedback and code changes closes without a human in the middle. Simon Willison’s Showboat takes yet another angle: agents construct Markdown documents with screenshots and command output that demonstrate what they built.⁸ As Willison puts it, “the job of a software engineer isn’t to write code, it’s to deliver code that works” — and just because the automated tests pass does not mean the software actually works. Showboat makes the agent prove it.

When I ask an AI to draft an email, the output needs to be scannable in seconds. If I end up rewriting it from scratch, I have not saved any time. When an agent researches a topic, showing me the sources alongside the summary lets me verify the claim without repeating the research. The products getting this right change the review surface from “read everything the agent produced” to “confirm the agent got the important parts right.” That is a different cognitive task, and it is much faster.

Review Is the Job

The question I keep asking, both for myself and when working with engineering teams: for every task we delegate to an AI, how long does it take us to verify the result? If the answer is “almost as long as doing it ourselves,” something is wrong. Either we need to change how the agent presents its work, or we need an intermediate check that removes the human from the loop entirely, or this particular task does not benefit from delegation.

A team that generates five approaches and verifies all five in an afternoon will outpace a team that generates one and waits a week for feedback. The bottleneck moved from “how fast can we build?” to “how fast can I tell whether this is right?”

I am experimenting with this myself. I have dozens of concurrent projects managed by AI agents, and I built a review app that presents each one as a card: what the agent did, what it needs from me, and a single input to give direction. No reading through terminal logs or git diffs. Just the decision surface, as clearly as I can make it.

It is early days, and I do not have all the answers yet. But I am increasingly convinced that the biggest gains in AI productivity will come from better interfaces for the humans reviewing what models produce, not from better models themselves.