Skip to content
The AI Agent Factory

Chapter 4. The Architecture in One Picture

Approved by
Panaversity ·
24 min read

The point

The last two chapters defined the AI Worker and gave it a rhythm: you set the work up in advance, it runs day and night, and you review it at times you choose. Look at what that means. On Monday at 6 a.m. the register run starts. On Tuesday at 11 p.m. a vendor emails. Nobody is at the keyboard, and between runs nothing of the worker is running. So everything a run needs must already be somewhere it can read, and everything it does must land somewhere a person can read at 9 a.m. Where is that? And when two of those places disagree, which one wins? This chapter answers with one picture, the same for every worker in the book.

After this chapter, you can draw any worker as six layers, each with one job. The company rents two of them: the channels where people reach the worker, and the runtime that does the work. The company owns three records: the KSoR holds what the worker knows, the SSoR records where each piece of work stands, and the DSoR is the door to the company's real systems. Memory sits on the line between rented and owned, because the AI vendor usually provides it, while what it remembers is about your company. Under the layers, a band names the Role Contract, the worker's one-page definition, which ties the layers together by saying what the worker may do with each one.

The architecture in one picture: six layers, one job each. Two you rent, three you own, and memory on the line between. At the top, the human principal sets the first 10 percent and answers for the final 10. Arrows lead to the two rented layers, drawn dashed because they are replaceable: Channels connect, where people and systems reach the worker, such as the chat app, email, team messaging or an API; and Runtime executes, the replaceable harness that reasons and acts, where a fresh machine for each run is enough. On a dashed line between rented and owned sits Memory, which remembers continuity, preferences, experience and where to look, but is never the authority. From the runtime, a line branches to the three owned layers, drawn solid: it reads the KSoR, which knows the governed, authoritative knowledge, such as policies, procedures, thresholds and definitions, each with its approved version; it records in the SSoR, which records the work as one case file for each matter: where it stands, what was tried, who approved what, and why; and it requests actions from the DSoR, which acts as the governed door to real systems: it checks who and what is allowed, acts, and keeps the evidence. Below the DSoR, arrows run both ways to the company's systems, such as accounting, bank and vendor records, which stay the authority on their data. A note reads: "When memory and a record disagree, the record wins. Rented layers can be replaced. Owned records come with you to any AI vendor." At the foot, a band names the Role Contract, not a layer: the worker's one-page definition, which says what it may do with each layer, through its knowledge sources, memory rules, actions it may request, runtime it needs, channels it serves, and evaluations it must pass.

Six layers, one job each. The picture this chapter draws.

With the picture you can answer four questions:

  • Which layers do we rent, and which must we own?
  • When memory and a record disagree, which wins? The record, every time.
  • Why does the worker need no permanent machine? Because everything that must last is kept in the records, so a machine borrowed for each run is enough.
  • If we switch AI vendors, or others start to use the worker, what do we rebuild, and what comes with us?

The picture also shows where each part of the worker's specification from the last chapter is kept, so that a run nobody watches can still find it.

The machine that is not there

The first chapter showed two ways to keep a worker going. Every run can get a fresh machine, wiped afterward, so between runs no machine exists at all. Or an always-on agent can have its own computer, which keeps running between conversations. Engineers have names for these two designs. In the first, no machine waits for you: one is borrowed for each run and handed back. That is a serverless design. In the second, the machine itself holds the worker's working state: its files, its open browser, what it was in the middle of doing. That is a stateful design.

Think of an office. A stateful worker has a private office: its papers stay on its desk overnight, ready in the morning. A serverless worker uses any free desk each morning. Everything it needs is brought to the desk at the start of the shift: its instructions, the files it works on, its notes from earlier runs, and the company's records. At the end, everything worth keeping goes back: the finished files, the history of the run, the updates to the records. Then the desk is cleared. That looks like a loss until you notice two things. A desk costs nothing while nobody sits at it. And nothing important is ever trapped in one drawer: when the desk is cleared, everything is still in the filing cabinet, where everyone who needs it can find it.

Look closely at that cabinet, though, because it has two kinds of drawer. Some things are kept in your account with the AI vendor: the task's instructions, your files and artifacts, its memory, its run history. They are durable, so they survive the cleared desk, but they are kept on the vendor's terms, and they are not the company's official truth. That was the fourth fact in the first chapter. Other things are the company's own records: governed, owned, and the place an auditor checks. Most of this chapter is about which thing belongs in which drawer.

That is the design choice the first chapter promised to explain. A worker needs continuity, not a permanent machine, and continuity can come from records. The serverless design asks this chapter's question, where is everything kept?, on every single run, because the desk remembers nothing. The stateful design can hide the question for months, until the machine is replaced, the AI vendor changes, or an auditor asks what the worker knew on September 1. Both designs can run a good worker. But on either one, the parts that must last belong in records the company owns, not on a machine it rents.

Two panels titled "The machine that is not there", subtitled "A worker needs continuity, not a permanent machine. Continuity can come from records." Left, "Stateful: a private office": one machine, always on, keeps the worker's state, drawn as a desk with papers left on it overnight. Its strengths: context ready at once, and an open browser mid-task. Questions to ask: how long do its files last, who else can read them, and what moves if we switch? A red box names the risk, state on a machine you rent: the question "where is everything kept?" can stay hidden for months, until the machine is replaced, the AI vendor changes, or an auditor asks what the worker knew on September 1. Right, "Serverless: any free desk, and a filing cabinet": a fresh machine for each run, wiped afterward, drawn as two empty desks for Monday 6 a.m. and Tuesday 11 p.m., each cleared after the run. A filing cabinet with two drawers sits beside them. The top drawer is your account, durable and kept by the AI vendor: instructions, files, artifacts, memory and run history. The bottom drawer is the company records, governed and owned, the official truth: KSoR, SSoR and DSoR. Arrows show that each run is loaded from the cabinet at the start and saved back to it at the end. A box names the gain, nothing trapped in a drawer: no machine sits idle between runs, each run is loaded from the cabinet and saves back to it, so nothing lives only on the desk, and the open question is which drawer each thing belongs in. A banner reads: "Both designs can run a good worker. On either one, the parts that must last belong in records the company owns, not on a machine it rents. The question that sorts any design: what survives, the machine or the record?"

A private office or a filing cabinet. What survives, the machine or the record?

The week below shows what happens without the picture: one worker, four failures in five days.

Why it matters

After September's month-end close, Brightline's AP Worker had a Role Contract and a rhythm, but no shape. Its instructions were kept in a shared project. The AP policy was a file someone had uploaded. Memory was switched on. Two connectors let it read the AP inbox and update the invoice register, and a scheduled task built the register every Monday. In the week of October 12, this setup failed four times.

Monday night: a wrong due date. Lakeshore Janitorial emailed to ask when invoice 5102 would be paid. The worker drafted the reply: Lakeshore's terms were Net 30, payment due 30 days after the invoice date, so the October 9 invoice was due November 8. In July, Maria, the office manager, had mentioned that Lakeshore was Net 30, and the worker remembered it. In August, Lakeshore moved to Net 15, and the vendor record in the accounting system says so. The worker never looked. The draft cited nothing, so it raised no flag, and it was approved and sent at the morning review.

Wednesday: a missed approval. Maria asked whether a $7,800 Midwest Packaging invoice needed approval from Dave, the controller. The worker said no: only invoices over $10,000 did. That was policy version 1, the copy from March in the project. Version 3, approved on September 1, lowered the limit to $5,000. But version 3 was in Dave's email, where the worker could not find it.

Thursday: a false approval. The worker built the payment run, that week's batch of payments, and marked a $3,960 Tri-County Freight invoice, number 5120, "approved by Dave." An email in the AP inbox, from a lookalike address, said Dave had approved it. The invoice needed no approval of its own, being under $5,000. But policy version 3 says Dave approves each payment run as a whole, a rule the worker could not reach either. He had not approved this one. The connector let the worker change the status, and nothing checked who had really approved. Maria found it at Dave's Thursday review, before any money moved.

Friday: the vendor question. IT asked whether the worker could move to the other AI vendor. Dave listed five things that would have to move: the project instructions, the policy upload, the memory, the scheduled task and the two connectors. He could not say which of them were the worker and which belonged to the AI vendor's product.

Monday's and Thursday's failures happened in runs nobody was watching, as a standing worker's runs usually are. A review at 9 a.m. can catch only what a run wrote down and cited, and these runs cited nothing. Wednesday's happened with Maria right there, and she could not tell either: the answer looked confident, and nothing showed which version of the policy it came from.

One week, four failures, one cause: the week of October 12, when each time the worker trusted the wrong source. A table with four rows and four columns: the day, what the worker trusted, what should have won, and the result. Monday, a wrong due date for Lakeshore invoice 5102: the worker trusted memory, "Lakeshore is Net 30" from July, so due November 8; a record should have won, the vendor record in the accounting system, Net 15, due October 24; it told the vendor a date 15 days late, and because the draft cited nothing, there was no flag. Wednesday, a missed approval on a $7,800 Midwest invoice: the worker trusted an old copy, policy version 1 in the project, approval only over $10,000; the approved version should have won, policy version 3, approved September 1, approval over $5,000; it told Maria no approval was needed, and it was, because version 3 sat in Dave's email. Thursday, a false approval on Tri-County invoice 5120: the worker trusted an email from a lookalike address saying "Dave approved it"; the system that holds approvals should have won, Dave's own approval of the payment run in the accounting system; it marked the invoice "approved by Dave", and he had not, but it was caught at Thursday's review and no money moved. Friday, can we switch AI vendors: Dave had a list of settings, the project instructions, policy upload, memory, scheduled task and two connectors; a picture of the worker should have won, showing where each part lives, who owns it and which part wins; IT got five items and no answer, because nobody knew which parts were the worker. The banner at the foot names one missing map, of location, ownership and authority. It says to define where each part lives, who owns it, and which source wins when records disagree.

The week of October 12. What the worker trusted, and what should have won.

Four problems had one cause. Nobody had drawn where each part of the worker is kept, who owns it, and which part wins when two of them disagree. Each failure is the same mistake in a different place: a memory trusted over a record, an old copy trusted over the approved version, an email trusted over the system that holds real approvals, and a list of settings mistaken for a worker. And each has a home in the picture. Monday and Wednesday are about what the worker knows, and which source wins: memory or a record, an old copy or the KSoR. Thursday is about what the worker may do, and where a real approval lives: the DSoR and the SSoR. Friday is about which layers you rent and which you own. The lookalike email would still have arrived. The picture stops it from becoming an approval. This chapter draws it.

Owner
Panaversity

On this page