Lesson 5.55.5 Iterate: change the part that failed

In everyday life. A recipe came out too salty. You cut the salt next time, not every ingredient.

To iterate is to improve a brief round by round. Read the first result as a test of your brief. Find its biggest problem, then ask whether your brief covered it. Say the worker used policy version 2, and your brief never named a version. That is a gap in the context. The table shows which part to change for each kind of problem.

A wrong fact taken from a source is a context failure. A claim that no source supports, or that must wait for someone's decision, needs a content rule.

Change that one part, run the brief again, and compare the two results. Changing everything at once hides what worked.¹

Every failure has a part. At 9:00 on Tuesday, October 20, Maria asks the AP Worker to draft a note to Lakeshore Janitorial about invoice 5131, which was due on October 15. Her brief names the outcome, who the note is for, an email draft, "Under 150 words, polite and plain" and "Save it as a draft and send nothing." At 9:10 the draft comes back, short and polite. It says Lakeshore "will be paid on Friday, October 23" and that its terms "are Net 45." But Dave does not approve the run until Thursday, and he has not yet confirmed the terms question. Which part failed? The outcome, context and format were met, and the worker sent nothing. The note said something it must not say, and that is a content rule, the first half of the constraints. At 9:15 Maria adds three content rules to the brief. They tell the worker to quote the invoice number and amount exactly as recorded, to say the invoice is under review without a payment date, and to leave out the terms until Dave confirms them. She runs the brief again and compares the two drafts. At 10:00 Dave approves the second one without edits. The outcome ended "Done when a saved draft says it is overdue and under review." The worker could check that sentence, but only Dave could approve the note. If the brief is good and the work is still weak, the brief is not the problem. Try a stronger model, if your app lets you choose one, or split the work into smaller stages.

Change the layer that failed. Before you change the brief, ask where the failure came from: the brief, the worker's definition, a skill, a source or a setting (5.1). If the same promise turns up in every vendor note, the rule belongs in the worker's definition, not in every brief. If a file was wrong, fix the source. Chapter 11 shows how to find the cause of failed runs.

Steer during a run. You can correct a worker while it is still working. This is called steering. Name the part that changed, such as "New constraint: hold both Tri-County invoices until Maria has called them", instead of sending the whole brief again.

Run saved briefs again after a model change. A line written for an older model can make a newer one do too much. One AI vendor found that its newest model checks its own work without being told to, so old lines that ask it to check made it check twice. Removing those lines lost nothing.² So when the model changes, run each saved brief again before you trust it. Remove only the lines written for the older model. Keep every rule that a policy or the worker's owner set.

Save your brief as a file, so you can reuse it. Then write each lesson into the saved brief. Maria's next version of the brief says "terms from the vendor record" because the last run taught her that. Stop iterating when another round changes little and finishing by hand would be faster.