Older AI desks were never checked against our new-desk checklist
1What happened
Our fleet has a checklist for setting up a new AI desk. An AI desk is an AI agent that runs one area of the business. The desks we built before that checklist existed were never checked against it.
One desk's job file named a tool connection for our ad account that was never connected. The file never named the tool the desk used instead.
The checklist had gaps of its own. It didn't check that a desk's tools work, or that the desk uses its skills.
2What we did
On Sat Oct 24 we checked all 32 live desks, reading only and changing nothing. For each desk we compared the folder it runs from, the tools its job file names, the tools it used in the last 7 days, and its saved memory.
We found 13 gaps. Among them:
5 desks had no saved memory.
1 desk ran from the wrong folder, so updates to its files never reached it.
6 desks used tools their job file never named.
Some tools named in job files were never used.
We fixed each gap once the owner said yes. Each desk's job file was rebuilt, and each desk read its new file back. The checklist now also checks that tools work and that skills get used.
What we got wrong first:
2 of our first hits were wrong. A shared rule already covered them, so we dropped them.
In the check, we first counted each desk's memory by its folder name, and the numbers were wrong. A desk's memory folder comes from its startup settings, not its folder name. We recounted from the running desks.
3How we know it worked
On Sun Oct 25 at 8 PM PT we checked three things:
the saved memory on each desk
that each rebuilt job file names the tools the desk uses
that the moved desk now runs from the right folder
All 13 gaps pass. One desk had only 1 saved memory file, so we looked closer. That count is right: the owner's past words for that desk already sit in its project files, so nothing was left to save.
4What you can take from it
When you add a checklist, run it on what you already have, not only on what you build next.
Compare a job file with what the desk does: the tools it used last week, not the tools it lists.
Count memory where the running desk reads it, not where you expect it to be.
