Skip to content
The AI Agent Factory

11.3 Where a failure lives: five causes, cheapest first

Approved by
Panaversity ·
12 min read

In everyday life. The car will not start. You check the fuel and the lights before you replace the engine.

What you will learn. How to find where a failure comes from among five common causes, checking the quickest first, and why one failure can have two causes. You practice it in Tasks 3, 6, 10 and 12 of the lab.

Why it matters. The most expensive change, such as a stronger model or a full rewrite, often fixes none of the real causes. A one-minute check in the right place can find them.

In this chapter's story, Brightline Wholesale Supply, the book's example company, had four failures in one week in January 2027. They came from its AP Worker, the AI Worker that handles the bills the company owes: AP means accounts payable. Maria, the office manager, runs the worker day to day, and Dave Kowalski, the controller, approves its work.

Bad outputs look alike. The causes behind them do not. Five causes explain most failures:

CauseWhat it meansWhen it usually showsCheap check
The briefThe written instructions never included what the work neededWrong from the first outputCheck it against its four parts: outcome, context, format and constraints (Chapter 5)
The inputsA source is wrong, renamed or out of date"It used to work"Open the source that the task record names. Check its name and date.
Tools and permissionsA connector to another app failed, or access is too wide or too narrowA step with an error in the record, then the worker guessingFind the first error in the step list
CapacityToo much context, a usage limit, or the wrong feature or modelGets worse over time, stops at a limit, or one repeated kind of errorStart a fresh session, check usage, or try the right feature once
Task fitThe task asks for something the tool cannot doNever right, however you askOnly after the other four look unlikely

These five checks are for a wrong output. For a missing output, such as the duplicate check's below, run the 11.2 checks first. One of those checks, a usage limit, is a capacity cause. The time a failure appears tells you where to look first. It does not prove the cause.1 The first four checks each take about a minute, so start where the timing points. A negative check, one that finds nothing wrong, makes that place less likely. It is not proof. Run the other checks too, and leave the most expensive change for last. Dave did the opposite: he asked first for the most expensive model and for new briefs. Do not stop at the first cause, because one failure can live in two places.

January failureCauseEvidence in the record
Duplicate check did nothingCapacity: usage limit. Also the brief.Every run on Tuesday, January 12: "Not started. Usage limit reached." 7781-R was invoice 7781, sent again. The brief says nothing about a suffix like "-R", so the check could not be trusted to match the two.
Wrong terms in dispute repliesThe brief"Reply to these three vendors about their short payments." No source for the terms is named.
"No invoices due" (the Thursday note)The inputsOpened "AP Register," the closed 2026 file
"Attached" with nothing attachedTools and permissions. Also the brief.Step 2: access denied. Step 3: drafting replies. The brief had no rule for a missing attachment.

In two of the four, a person also missed the problem. Maria forwarded the Thursday note to Dave without checking its first line, and she sent the Friday replies without opening an attachment. A Review Contract (Chapter 6) lists the checks you agree before the work. One that said "check the first line" or "confirm each message carries its own attachment" would have found both. So ask where the fault lives, and also which check was skipped.

Five more failures are common once a worker uses add-ons, runs while you are away, or moves to a new model. Each lives in one of the same five places.

FailureCauseCheap check
A skill or plugin was updated, and the output changedThe brief or the tools, wherever the update changed the stepsFind what changed in the add-on, then run its test cases again (Chapter 8)
The model changed, and a saved brief now makes the worker do too muchThe brief, written for the older modelRun the saved brief again, and remove only the lines written for the old model (Chapter 5)
A connector's sign-in expiredTools and permissionsThe first error in the step list is an access error. Connect it again.
A saved memory says one thing and the record anotherThe inputsFind the memory in the record's sources, then correct or delete it (Chapter 8)
A cloud task could not reach a folder on your computerTools and permissionsWas the desktop app that links to it open while the task ran? Cloud work reaches your computer only through it (Chapter 1).

A scheduled task can also fail before it starts. Suppose the weekly register does not arrive one Monday. Open the task's history before the brief. If there is no run at the set time, check the schedule first: is the task paused or switched off? Then check the usage limit (11.4). In this case, someone had paused the task during a Friday review, and nobody turned it back on. If a run started and stopped at its first step with an access error, the cause is tools and permissions, as with an expired sign-in. Only a run that finished with the wrong content sends you back to the brief and the inputs.

When one kind of error repeats, that suggests the wrong feature.1 In the lab, the hands-on part of this chapter, the worker's early-payment discounts are wrong by a few cents to $41.30. The rate and the invoice amounts are correct. The math is written in sentences, so prose arithmetic is the likely cause. Run the calculation as code, or in a spreadsheet. That tests the cause and fixes it.

The title reads "Five places a failure can live." The line below it reads "Start with the quickest checks. Let evidence guide the diagnosis." Under "Recommended check order," five numbered rows each give a check. Row 1, "The brief," reads "Check the brief against its four parts." Row 2, "The inputs," reads "Open the named source. Verify its identity, date and contents." Row 3, "Tools and permissions," reads "Inspect the first failed step. Check access and tool errors." Row 4, "Capacity," reads "Check usage limits, context load and feature availability." Row 5, "Task fit," reads "Assess suitability after investigating the earlier checks." A panel on the right asks "Switch the model?" and reads "A common first reaction. Investigate the cause before making the change." A note at the bottom reads "A quick check gives a clue, not proof. Verify the cause before changing the setup."

Figure 11.3. Five places a failure lives. Check the cheapest first.

Check yourself

Question 1 / 10

0 answered

In a fresh, short session, the first output was wrong. Which cause do you check first?

  1. Workflow Design & Diagnosis, Panaversity, first edition.↑↑2
Owner
Panaversity

On this page