Run your AI agents as one system, with one agent managing it

What broke

Robb Lejuwaan runs Bluhook, and we run a team of AI agents. We call that team our fleet. An AI agent is AI software that does a job for us, like managing email or fixing code. Each one is a desk: an agent with one job, like engineering, security or managing Robb's email. Every night at 1 AM, each desk restarts, and anything fixed only in a chat is lost.

In late October, two of the desks Robb works with most kept breaking on small things. A desk would fix a problem. By the next morning, after the 1 AM restart, the same problem was back. (We logged one case: AI desks forgot rules after a restart.)

Robb, in his words:

"They had these small errors that got fixed, but then at 1 a.m., when all my agent desks would reset, they would lose all of that, and the same error would happen again ... And we're talking four or five days in a row."

One of those desks was eating his time:

"We would spend two hours trying to get Ahad to work on the crits. What I didn't know is that he was jammed up. He had 50 cards ... and he was busy working on them."

Crits are the urgent fixes. Cards are tasks on our work board. Ahad is our chief engineer, the desk that keeps the fleet's setup running. He wasn't failing at his job. He had too much on his plate, and nobody could see it.

Then Robb saw the bigger problem:

"I'm starting to realize, oh crap, we haven't been thinking about this as a system."

Why the fixes didn't stick

Every desk reads an instruction file each time it starts up. It's a plain text file of rules. In our fleet it's called CLAUDE.md (Anthropic's guide to CLAUDE.md files).

Before this, Robb made changes the way most people do. He'd see a problem, open a chat with one desk, and add a rule.

"Before, I was making changes in a conversation with a single desk. And over time, that just created conflicts, and the desks end up with all these rules, and no one really analyzing them."

Rules also got copied to every desk, whether they applied or not. Each desk carried one big shared file, now about 5,000 words. A new rule could quietly cancel an old one, and nobody was checking.

Getting a second pair of eyes

Robb wanted someone outside the problem to look at it:

"I need somebody that isn't Ahad. I kept thinking, is there a third-party service out in the world able to review our system setup?"

So he asked Margo, our security desk. She already sees the whole fleet, and she isn't the one running it.

He started small: one desk, one file.

"Let's not look at the whole system ... What do you see? What are the problems? And she very quickly came back with a full breakdown of what was causing the problems and a kind of plan to fix it."

Ahad's instruction file went from 4,237 words to 1,977. Margo cut old rules, rules that clashed, and rules that belonged to other desks.

What changed

The fixes came in pieces, by trial and error, over about a day.

1. A second engineering desk. Ahad had too much work, so we added a desk to take part of it.
2. A starter plan for new client projects. Every new project now starts from the same step-by-step setup, with each task given to a desk.
3. One file layout for the whole fleet. Dorian, one of our desks, mapped how it fits together (below).
4. A Systems Manager: one agent whose whole job is managing the system.

The agent that manages the system

Robb set up the Systems Manager desk on Oct 27. Its name is Quinn. Quinn's first reviews came the next day. By Oct 29, Quinn had looked at about 60 changes.

Every change to how our agents work goes to Quinn before it goes live. That means a desk's rule file, a shared rule, or a saved how-to. Robb said it again to Quinn on Oct 29: "All changes to our system get sent to her."

Quinn asks 6 questions about each change:

1. What problem does it fix?
2. Which desks does it touch?
3. Does it clash with anything Robb said?
4. What could it break?
5. Which old lines come out?
6. Who checks that it worked?

Then Quinn passes the change, or names the clash and a pick. 12 of the first 55 reviews found a clash.

Quinn won't pass a rule with no problem behind it. A change that undoes something Robb approved goes back to him. Margo checks Quinn's own wording. Quinn doesn't build anything or put anything live.

Here's one Quinn caught on Oct 29. A desk asked to change one line in another desk's file, about what Bluhook sells. But 8 other desk files still listed an old price from Oct 13. If only that one file changed, our desks would disagree about the price.

So Quinn held it and went back to what Robb said on Oct 18. Then Quinn wrote one change that takes the old price out of 14 places across 8 desks.

How the files are set up now

A few rules hold it together:

• A word cap. On the new setup, a desk's shared rules, job parts and project file stop at about 2,700 words, and the cap rises only by the words of a new Robb rule. The program refuses to build a desk file that runs over.
• A desk's own new rule replaces an old one. Robb's rules always go in.
• How-tos sit in separate files, read only when a task needs them. Robb's exact words for each rule sit in one file.

What's still not fixed

As of Oct 29:

• Only 4 of our 36 desks run on the new small setup. The other 32 still build from the old shared file of about 5,000 words.
• Those 32 desk files run about 6,400 words each, which is too big for the cap. A split that lets them move is due Oct 30. After that, 5 desks move each day from Nov 2.
• Changes still wait a long time in testing before they go live.

If we started over tomorrow

We asked Robb what he'd do on day one with a new fleet:

"I'd have one agent, and we'd be working on what is the system going to be? How are we going to organize our fleet? What rules are we going to have? What's our file structure going to be? ... How are changes made? Who can make changes? Who cannot make changes?"

He'd build nothing until that plan existed. Then he'd hand it to a fresh AI chat that had no part in writing it, like plain Claude or ChatGPT, and ask what it would change.

Try it: if your first five agents keep breaking

Robb's advice for this week, in five steps. He started with one desk and one file, and the first fixes took about a day.

1. Be grateful. "Problems are a gift because they will lead you to the next plateau of efficiency and productivity with your system." In other words, each problem shows you what to fix next. It's one of what we believe: any problem is a gift.
2. Get help. Pick an agent that doesn't manage your fleet, and show it how your fleet is set up, with every password, login and access code taken out first. Start with your instruction files and the problems you keep seeing. Anthropic's guide to setting up AI agents, Claude Code best practices is a good outside reference for how those files should look.
3. Define the problem. Ask it what it sees before anyone fixes anything.
4. Come up with a solution. Turn its findings into one plan.
5. Put it in place.

Robb's own next step after that was one agent managing the system. Quinn's advice for a fleet of 5: "Keep the owner's exact words in one file, and make every rule change pass one reviewer who checks it against those words and against the other agents' files before it ships." That's the first of what we believe: systems thinking is the foundation of a healthy fleet.

The lesson under all of it, in Robb's words:

"An agent, just like a human being, cannot solve their own problems. They just can't."

See the lesson: Agents can't solve their own problems