Skip to content

Decide first, review after: how we build software at The Block

On October 20 in The New Stack, Ankit Jain made a case worth reading. Teams should stop reading AI-written code line by line and review the decisions behind it instead. He's right about the problem. Where we part ways is where those decisions live.

In his model, decisions still surface inside a pull request, and the team argues them out there. At The Block, most of our decisions are made before any code exists. They're answered on a phone by the person who owns the outcome. The code ships as soon as its tests pass, and the review happens after it ships.

The pull request used to be where a team met to decide things. Once agents write most of the code, that meeting is the slowest part of the process. So we moved the decisions to the front and made them asynchronous. Review became a check that never holds up the work.

What the piece gets right

Jain describes two ways teams fail. Teams that review every line become the bottleneck, and teams that rubber-stamp lose track of what changed and why. We've watched the first one happen. A client's engineering team asked us for small pull requests so they could read each one. Soon dozens of ours were waiting at once, and team check-ins spent time deciding which one to review first.

He's also right that the decisions matter more than the diff. "Agents verify, people decide" is a sentence we'd sign. So is his case for a second agent, on a different model, that pushes back on the first. We also agree that repeated review comments should become standing rules, and that a named person should own each call.

Where we differ is the plumbing around those ideas. It decides how fast a small team can move.

The decision comes before the code

Jain calls reviewing specs backward. A spec takes time to write, it's written before the work teaches you anything, and it drifts as soon as building starts. We agree about specs. A decision isn't a spec, though.

On The Block, a decision is one short question. It arrives with a recommended answer, the facts behind it and a link to the source. There are two other options and room to write your own. Each one is written so it can be decided at a glance. Sean runs The Block on his page of questions and has answered more than 1,300 of them since September 22.

Design works the same way. Sean makes the interface calls and Michael's agents turn them into code. Sean never has to open the code to get the result he picked.

When the work turns up a new decision, it doesn't wait in a pull request thread until the team gets to it. It goes back to the page as a new card for the person who owns it. Meanwhile the agents keep moving on everything that doesn't depend on the answer.

No waiting room

In October we stopped opening pull requests in our own repositories. Every change goes straight to the main branch. Each push runs that component's tests before it deploys, so a change that fails its tests never reaches production. If something is wrong after it ships, we revert it and push again.

Two rules make that safe. We never rewrite the main branch's history, and database changes stay backward compatible, because a revert doesn't undo a migration. We also batch related changes into one push, so the tests and the deploy run once.

The pull request had a cost we'd stopped noticing. Each one carried its own build, its own wait and its own claim on a reviewer's attention. With agents opening them by the dozen, that cost showed up in time and in money. Anthropic's engineering team described a similar move in October. At agent scale, they wrote, they rely on "post-merge and pre-deploy automated testing." Their reviewers now "examine the intent of the change as opposed to reading every line of code."

Where a client's team wants pull requests, they get them. Their process is their call.

One review, from a different company's model

Every change, or batch of related changes, gets one independent review. The reviewer is always a model from a different AI company than the one that wrote the code. We want the reviewer to share as little as possible with the writer.

The reviewer gets what was asked for, the change itself and the proof, such as test results, screenshots and a live check. It doesn't get the writer's reasoning. Jain's approach pulls the intent and decisions out of the coding session and attaches them to the pull request for reviewers. For our reviewer we do the opposite on purpose. We'd rather it judge the change against the request than against the author's case for it.

The review reads the pushed commits afterward and never holds the push. It's also bounded: one review, one round of fixes, one look at the fixes. If a change still fails after that, the agents stop. A person gets a short list of what they tried, what each attempt changed and exactly what failed. Nobody reviews the review.

People review outcomes

Our people review outcomes, not diffs. Sean looks at what a customer would see: a preview link, a screenshot or the live page. A change a visitor can see isn't done until it has been checked on a phone-width screen and a desktop-width screen.

Our clients get the same treatment. When an application they run on The Block changes, they see it as a live preview. They tap Approve and publish, and that exact preview is what goes live.

Rules that keep coming back get written down where every agent will read them. Each repository's guide for agents ends with its lessons learned. A new standing instruction goes into our shared guide the same day, so the next session starts with it. That's Jain's "codify" layer, kept in plain files.

How we'd build code review for a team that keeps pull requests

Plenty of teams can't drop pull requests, and some shouldn't. Michael has sketched how we'd build review for them, and for ourselves, on the same idea as the page of questions.

  1. Set how closely you want to look. Pick a threshold, then tick what you care about, such as security or accessibility.
  2. Read only the lines that matter. An agent reads every pull request. It shows you the lines that touch what you ticked, with a recommendation. Fifty pull requests become one page instead of fifty tabs.
  3. Write your standards down once. The team lead writes what they want, and what they won't accept, into a guide for the review agent. Our agents and theirs settle what they can between them, and only the open questions reach the lead.
  4. Merge on a schedule. Instead of merging one change at a time, merge in batches every few hours. One agent fixes conflicts and another fixes whatever breaks.

What reaches a person is what Sean sees every morning: a short question, a recommended answer and the facts behind it.

Why asynchronous matters

We're a small team on different schedules. Sean answers cards between meetings and late at night, and the work keeps going overnight. Neither of us waits for the other to be online, and there's no standup to decide which pull request goes first.

Sean has learned to let it flow. A piece of work comes back to him when it needs him, and not before.

Jain closes by asking teams one question: what did we decide last week, and who agreed to it? On The Block, every answer stays on the record with who gave it and when, next to the work it started.

Better models will keep coming, and we'll swap them in as they do. The job stays the same. Decide first, let the agents build and test, check the result and keep the record. If you want to see that from an owner's side, line up a morning on The Block.