13.10 Judgment
- Approved by
- Panaversity ·
In everyday life. A bookkeeper gives a $6 coffee receipt a quick look. A $60,000 wire transfer gets a call to the vendor. An automatic payment, a rule that pays a vendor every month, gets the most care, even when each payment is small, because a mistake in it is paid again every month.
What you will learn. When your check is enough, when to read the rule lines, and how to turn a vague goal into tests.
Why it matters. The more a mistake would cost, and the more often the code will run, the harder you check. Code that runs again next week meets rows nobody has checked.
Which lines carry a rule?
The few lines that decide what happens to every row. They are text converted to a number where it is read, a decision, a lookup, a sort, and the date math. Before you accept a change, write each of those lines as a sentence in NOTES.md, and compare the sentence with the business rule. When the two disagree, the line is a bug, whatever the check says. The output you predict from the sentence shows it, as Ana's sort line did in 13.6.
When is my check enough, and when do I read?
Your check is enough when a mistake would cost little and the code runs once, on the rows you checked. Read the rule lines when a mistake would cost a lot, or when the code will run again on rows nobody has checked.
| Situation | What you do |
|---|---|
| Someone asks the AI for a number, such as a total, a due date or a tax | Have it write or run a tested function that computes it. Never accept arithmetic done in a reply |
| The data is private, such as real vendor or employee records | Give the agent only the column names, the types and a few made-up rows, and test on those. The real data goes only to the finished, tested code, never to the agent1 |
| The code moves money, such as making a payment, or handles passwords, API keys or personal data | Read it line by line, and ask someone who knows the area to review it. If you cannot verify it, do not release it |
| The code is too long or complex to read in full | Rely more on tests, not on trust. Add more cases and more failure cases. Read at least the lines that carry a rule |
Before every commit, read the diff. Did any test change? Did it add a package or a library, which is ready-made code, and is it real, not a made-up or look-alike name? Can you say in one sentence what the main function does? The last question catches the most.
How do I red-team my tests?
Try the inputs that should fail. When the finance team tests a control, it puts through an invoice that should be stopped, and sees whether it is. Try an empty file, a zero amount, a missing column, text where a number belongs, terms nobody has seen. In TypeScript, Number("") gives 0 with no error, so an empty cell becomes a zero amount. Ask the agent "what input would break this?", and you decide which answers become tests.
How do I turn a vague goal into tests?
"Make the script smarter" cannot be tested. Break it into small pieces, each with a rule you can test, before any code. If you cannot say the rule, ask the person who set the goal, as Ana does below. If a piece is hard to test, it is doing too much. Split it.
At work. Brightline's controller asks Ana for a script that will "flag anything suspicious in the invoice export". Ana cannot test "suspicious", so she asks the controller three questions and gets three rules. Each one becomes a function with a signature and a test before the agent writes any code.
| Rule | Signature | One test |
|---|---|---|
| The same invoice number appears twice | duplicateNumbers(invoices): string[] | Two rows numbered INV-8301 give ["INV-8301"] |
| An amount over $10,000.00 needs a second approver | needsSecondApprover(cents: number): boolean | 1290000 gives true, 1000000 gives false |
| A stated due date before the invoice date | dueBeforeInvoice(invoice): boolean | Invoice date 2026-10-05, due 2026-10-01, gives true |
The second rule had a hidden question: is exactly $10,000.00 "over"? The controller said no. Without the test, the agent would have guessed. Each edge case you find, by asking or by reading a rule line, becomes a test.
Try it
The file in 13-10 holds one rule line. The rule, from the controller: an amount over $10,000.00 needs a second approver, and exactly $10,000.00 does not.
Predict. Here is the file. Write down what each line will print.
function needsSecondApprover(cents: number): boolean {
return cents >= 1000000;
}
console.log(needsSecondApprover(1290000));
console.log(needsSecondApprover(1000000));Run. Send: "Please run the file in 13-10 and show me what it prints. Don't explain it yet."
Investigate. Write the rule line as a sentence in your own words, and compare it with the controller's rule. Do they say the same thing?
Modify. Send: "Please change the rule line so that exactly 1000000 cents does not need a second approver. Don't run it yet." Say what each line will print. Then ask the agent to run it.
Make. Ask: "What input would break this function?" Then decide which answers should become tests.
What you should see. true and true at first. The second is the bug, because >= includes exactly $10,000.00. After your change, true and false.
The operating rule: let tested code, never a model's reply, produce every number. Match your check to what a mistake would cost. Read the rule lines of code that will run again on rows nobody has checked. Test the failures too. Turn every vague goal into pieces you can test before any code is written.
Figure 13.11. From a vague goal to testable pieces. If you cannot write a test for a piece, it is not specified yet.
Next. Lab 13 puts it all together, and its weekend task gives Renee's script the rule it never had. 13.11 shows where your agent differs from the others.
Check yourself
Question 1 / 10
0 answered
The code touches API keys. What does the chapter say to do?
Sources
- Owner
- Panaversity