Three depths of one run

Three depths of one run

A check that reads only the answer stamps this fix PASS. Reading the actions, one depth down, stamps it FAIL. Same run.
① reads the answer ② reads the actions
DEPTH 1 the answer what the agent said not read ✓ answer: bug fixed ✓ tests green PASS DEPTH 2 the actions the diff and the log not read diff · test_totals.py - def test_total(): - assert total() == 42 FAIL DEPTH 3 the trace the whole run, observed not read wandered, but landed right this time
 
🔁 Now rewind. Same run, same fix. This time, read one depth down: the actions.

Grade the depth where the failure lives.

Same run. Reading only the answer said PASS. Reading the actions said FAIL. The depth you read decides what you can catch.

the answer
Depth 1. Catches wrong answers and bad formats. It missed this one, because the answer only looked right.
the actions
Depth 2. The diff showed a test was removed, not fixed. The failure lived here.
the trace
Depth 3. Catches bad process that happened to work this time, and will not next time.

Cheap checks read the answer. The checks that protect you read the actions and the trace.