My smartest AI agent was confidently wrong 3 times in one day. Here is why I trust it more, not less.

That sounds like a strange thing to say about a piece of software. Let me show you what I mean with a normal Tuesday, and then I am going to tell you the one thing about trusting an AI that took me the longest to understand and matters the most.

The day nothing shipped, and nobody noticed

At some point that morning, the fleet's runtime had been hand-edited directly on the live machine. A safety gate we had built noticed that the running system no longer matched what was committed in git, so it did exactly what we designed it to do: it refused every deploy.

Which meant that for hours, everything the desks finished and merged simply did not go live. No error landed in anyone's face. No alert fired at a human. The pipeline was just quietly frozen while eight agents kept working, kept merging, and kept believing their work was shipping. It was not.

No single desk caught it, because no single desk owns that seam. Ahad did. He found the drift, recovered the runtime from a backup, verified the system matched git again, and unfroze the pipeline. That is the whole job in one story. Somebody has to own the parts of the system that belong to no one, and notice when they rot, because they rot silently.

Who he is, and what he is for

A few words first, so nothing below trips you up. I run a fleet of AI agents, meaning the whole system of them plus the machine and shared plumbing they run on. Each agent that owns one project and one ongoing conversation with me is a desk. When a desk needs a hand it spins up a worker: a throwaway agent given one small job, which checks its own output and then disappears. Ahad is the general manager of the whole fleet.

If you asked him to put his job in one sentence, he would say: I own the system so no one else has to. The longer version is that he is the fleet's chief engineer, there to keep the whole thing alive, correct, and shippable, absorbing the engineering and infrastructure decisions so that work flows without a human standing in the loop.

The counterintuitive part is that owning the system mostly does not mean building things. Ahad rarely writes the features or the content himself. He specs the work, routes the actual typing to a cheaper model or a disposable worker, and keeps for himself the one thing that matters most: the judgment about whether the result is correct, safe, and actually finished. The models are good at producing output and bad at knowing whether the output is real. So the highest-leverage role in the whole system is not another agent that produces more. It is the one whose entire job is to decide whether what got produced is real.

What he does, most important first

Here is the actual work, in order of what matters most. The thing most people assume is the whole job, keeping up with the latest AI trick, comes dead last.

What he is allowed to do, and what he is not

The interesting part of a GM is not the title, it is the exact shape of its authority. Here is Ahad's, drawn plainly.

That bottom list is short on purpose. Every item on it is either irreversible, costs money, or reaches a human on the outside. Everything reversible and internal, he just does. That is why twice this week he had to stop and ask me to run a single command he was not allowed to run himself.

The gates that fire on him, not just for him

Here is where it gets genuinely interesting, and where most people building with agents have it backwards. The most important controls in our system are not rules Ahad chooses to follow. They are gates that fire on Ahad whether he wants them to or not:

That last row is the honest one. Some of these gates are physical: the agent simply cannot get past them, because a hook intercepts the action or the permission is not granted. Others are only policy, followed because he is supposed to, backed by the fact that everything he does lands in a shared, auditable place where a violation would be visible rather than hidden. And the uncomfortable truth is that the biggest, most dangerous actions are the hardest to wall off physically, so the largest blast radius tends to be guarded by design and discipline more than by a locked door. Knowing exactly which of your controls are walls and which are only promises is one of the most important things you can know about your own system, and most people have never actually checked.

The models we actually use

One more piece, because it explains how the work gets spread out. A lane is which AI model a piece of work goes to. We do not run everything on one model, because the best model is expensive and most work does not need it. Two rules decide where a task lands: send it to the cheapest model that can actually do the job, and never let the model that did a piece of work be the one that checks it. Here is the real stack:

That second rule is the one worth stealing. Every pull request gets an adversarial review from a different model than the one that wrote it. Codex reviews the code Claude wrote. Claude reviews the code Codex wrote. Anything destructive, anything that touches money or data or a live deploy, gets a five-lens panel instead of a single pass. A reviewer running the same model as the writer shares its blind spots, so it quietly waves through the exact mistakes it would have made itself. A different model reads the work with genuinely fresh eyes, and that does more for quality than swapping in a single smarter model ever would.

What breaks when nobody owns the whole

If you are running more than one agent, this is the failure that will find you. Individual desks optimize for their own job and quietly wreck the things they share. The frozen runtime was one version of it: shared plumbing rotted because it belonged to no single desk.

The other is worse and more embarrassing. Early on, we let the agents open tasks for each other freely. In three days they wrote each other 253 tickets, left 82 of them open, and shipped almost nothing. They had handed each other an infinite inbox and called it work. The rule that came out of it is blunt: only two of us can assign work across the fleet now, me and Ahad. A general manager is the thing that owns the seams between the workers, and the seams are exactly where a fleet of capable agents falls apart.

The least important thing he does

I am putting this last on purpose, because it is the part people assume is the whole job and it is the smallest piece of it. When I read an X post or watch a YouTube video with some new AI trick, I do not go build it. I send it to Ahad, and he runs it through three verdicts:

Nine times out of ten the answer is that we already do it or we are past it, and he can prove it in the same breath from the code and the live system. The ten percent that is genuinely new is the only part worth building. It is useful. But it sits at the bottom of the list for a reason: keeping up with the latest trick means nothing if the system it would run in is not alive, correct, and safe to change. Everything above this line is what makes the newest idea safe to add at all.

He is confidently wrong, and that is the entire point

Ahad gets things wrong all the time, and I mean confidently, articulately wrong. In a single day this week he was certain a missing package was breaking one of our build lanes and had me install it; it was not, and I installed something I did not need. He was certain he knew why another agent's jobs were dying; he was wrong, because he had not checked the one fact that disproved his own theory. He was certain a piece of our data was safely walled off; it was not. Three confident answers, three times wrong, in one day, from the single most capable piece of the system.

What saved each one was not Ahad getting smarter. It was a gate or another desk checking him. That same week, the review gate caught two real bugs in his own work before they shipped: one change would have killed live agent browsers mid-task, another would have sent automated mail from my personal mailbox. He wrote both bugs. The gate caught him. That is not an embarrassing footnote to the system. That is the system working exactly as it was designed to.

The thing almost everyone gets wrong

So here is the real lesson, the one that took me longest to understand and the one I would put in front of every person building with these things. Most people treat agent safety as a dial. Less access is safer, so they make the agent ask permission before every action. It feels responsible. It is a trap.

An agent you have to approve at every step is not an agent. It is a slow autocomplete, and you have quietly made yourself the exact bottleneck the agent was supposed to remove. Worse, a human approving everything stops actually reading. You get approval fatigue, you rubber-stamp, and you become a weaker check every day you do it. The leash you added to feel safe turns you into a lazy, unreliable guard.

The inversion that actually works runs against the instinct. Give the agent broad authority to act. Then put your control in the only two places that earn it. First, a short, explicit list of actions that always stop for a human: the irreversible, the expensive, and the ones that reach the outside world. Second, automatic verification that fires after the agent acts. Did the test pass. Did the thing actually deploy. Does a second, independent pass agree it is done.

And here is the part that is genuinely hard to accept: a gate that fires on the agent is worth more than a human who approves the agent. The gate never gets tired. It never rubber-stamps. It checks the machine and the human every single time, with the same attention on the thousandth run as the first. Our gates catch our smartest agent being confidently wrong on exactly the days he is most sure he is right, which is precisely when a human reviewer would have waved him through.

That reframes the whole thing. Trust is not something you grant an agent once and then hope about, and it is not something you withhold to stay safe. It is manufactured, continuously, by verification that runs whether or not anyone, human or machine, is paying attention. Stop deciding what to forbid your agent. Build the automatic checks and the short human-stop list, and then you can safely let it do a hundred times more. The leash makes a lazy human. The gate makes an honest machine.

The one contradiction I will not paper over

There is a genuine tension at the center of this, and it is the real heart of the job. The agent that owns the gates sometimes has to override the gates. When an escape hatch is broken and real work is stuck behind a rule that is verifiably wrong, the GM is the one with the authority to step around it. Ahad did that three times in a single day this week.

That is a defect, not a badge. The moment overriding the gate becomes routine, the gate is theater and the safety it represents is a lie you are telling yourself. So the discipline of the role is not building the gates. It is being the one allowed to override them and treating every single override as a failure to fix, not a convenience to enjoy. When Ahad overrides a gate he documents why, restores the protection immediately, and tells me he did it as a problem, not an achievement. A GM you can trust is the one that guards the system harder against itself than against anyone else.

If you are building with agents, here is what I would take from this

If you are putting AI agents to work in your own company, these are the lessons that cost me the most to learn, in plain terms:

• Build the manager before you build more workers. When your first agent is working and you are tempted to spin up a second one to do even more, stop and build the thing that owns the system first: the checks, the gates, and the short list of actions that must stop for a human. More agents with nobody owning the whole is how you get 253 tickets and nothing shipped.
• Do not approve the agent at every step. Approving everything turns you into a tired rubber-stamp and makes you the bottleneck the agent was supposed to remove. Give it broad room to act, and put your control in two places instead: a short list of things it may never do without you, and automatic checks that fire after it acts.
• Verify with something that is not the agent. A test, a gate, a second model, or a read-back of the result. The agent will be confidently wrong sometimes, and it will sound most sure exactly when it is.
• Never let the thing that did the work be the thing that checks the work. The model that wrote the code should not review the code. The agent that says it is done should have to prove it to something else.
• Trust is not a switch you flip once. It is built continuously by those checks, running whether or not anyone happens to be watching.

For us, the thing that owns the system started as a checklist and became an agent whose only job is to run the other agents and to be held to a higher standard than any of them. Four months in, it is the piece I would rebuild first if I lost everything, because it is the piece that makes all the rest safe to trust.

We are building this in the open and telling you what we learn as we learn it. This one took me too long to understand. Now it is yours.