The Complete Feedback Loop
James sits down for the capstone. He has his tests written, his terminal open, and a blank prompt waiting. No templates. No guided steps. Just the process he has been learning: prompt, test, read the diff, decide, repeat.
Emma watches from across the room. She has been stepping back a little more each lesson. In Lesson 1 she explained the funnel pattern. In Lesson 2 she pointed out the regression. In Lesson 3 she showed him how to read diffs. Now she says one thing.
"Verify before you trust."
James nods. He knows what she means. Even when all tests pass, read the code. Check the types. Look for dead code. Do not assume the AI got everything right just because pytest says green.
This lesson brings together everything from the chapter into one exercise. You will write prompts, run tests, read diffs, and make judgment calls. If any step feels shaky, go back to the lesson that covers it. The goal is to practice the full cycle, not to memorize it perfectly on the first try.
The Capstone: search_notes from Scratch
You will build search_notes from scratch using the full iteration process. Pretend you have never seen the function before. Your only inputs are the Note dataclass and the tests.
Setup
Create a file called test_search.py in your SmartNotes project with these tests:
from dataclasses import dataclass
@dataclass
class Note:
title: str
body: str
tags: list[str]
word_count: int
def test_search_exact_title_match() -> None:
notes: list[Note] = [
Note(title="Python Basics", body="Variables and types.", tags=["python"], word_count=3),
Note(title="Cooking Tips", body="Use fresh herbs.", tags=["cooking"], word_count=3),
]
result: list[Note] = search_notes(notes, "Python")
assert len(result) == 1
assert result[0].title == "Python Basics"
def test_search_case_insensitive() -> None:
notes: list[Note] = [
Note(title="Python Basics", body="Variables and types.", tags=["python"], word_count=3),
]
assert len(search_notes(notes, "python")) == 1
def test_search_in_body() -> None:
notes: list[Note] = [
Note(title="Cooking Tips", body="Use fresh herbs.", tags=["cooking"], word_count=3),
]
assert len(search_notes(notes, "herbs")) == 1
def test_search_no_matches() -> None:
notes: list[Note] = [
Note(title="Python Basics", body="Variables and types.", tags=["python"], word_count=3),
]
assert search_notes(notes, "JavaScript") == []
def test_search_empty_list() -> None:
assert search_notes([], "anything") == []
def test_search_empty_term() -> None:
notes: list[Note] = [
Note(title="Python Basics", body="Variables and types.", tags=["python"], word_count=3),
Note(title="Cooking Tips", body="Use fresh herbs.", tags=["cooking"], word_count=3),
]
assert len(search_notes(notes, "")) == 2
Save the file. Commit it so you have a clean baseline for diffs:
git add test_search.py
git commit -m "Add search_notes tests"
Round 1: Your First Prompt
Write a prompt at whatever funnel level you choose. Here is a starting point if you need one, but try to write your own first:
Write a Python function called search_notes that takes two parameters:
notes (list[Note]) and term (str). Return a list[Note] containing every
note whose title or body contains the search term. Note is a dataclass
with fields title (str), body (str), tags (list[str]), word_count (int).
Paste the AI's output into a file called search.py. Run:
uv run pytest test_search.py -v
Record the results in your tracking table:
| Test Name | Round 1 | Round 2 | Round 3 |
|---|---|---|---|
test_search_exact_title_match | |||
test_search_case_insensitive | |||
test_search_in_body | |||
test_search_no_matches | |||
test_search_empty_list | |||
test_search_empty_term |
For each failure, note the Error Taxonomy category (Omission, Misinterpretation, Logic error).
Round 2: The Re-prompt
Look at your failures. Write a re-prompt that targets them specifically. Include the pytest failure output. Before sending the re-prompt, save the current search.py so you can diff later:
git add search.py
git commit -m "Round 1 output"
Send the re-prompt. Paste the new output into search.py. Run pytest again. Update your tracking table.
Now read the diff:
git diff search.py
Check: did the AI change only what you asked for, or did it rewrite more than necessary? If a previously passing test now fails, you have a regression. Note it in your table.
Round 3: The Judgment Call
Apply the 30% heuristic to any remaining failures:
- If the fix is less than 30% of the function (one or two lines), open the file and fix it manually. This is faster.
- If the fix is 30% to 70%, write one more re-prompt with surgical instructions.
- If more than 70% is wrong, start over with a better prompt.
After your fix (manual or AI-generated), run pytest one final time. If all six tests pass, commit:
git add search.py
git commit -m "search_notes: all tests passing"
The "Verify Before Trust" Habit
All tests pass. You are not done yet.
Open search.py and read the function. Run through the quality checklist from Lesson 3:
| Check | Pass? |
|---|---|
| All variables have type annotations | |
| Function has a docstring | |
| No commented-out or dead code | |
| Variable names describe their purpose | |
| Style matches the rest of your project |
If any check fails, fix it now. These are quick manual edits. Do not re-prompt the AI for a missing type annotation or a vague variable name.
This habit, verifying code quality after tests pass, is what separates someone who uses AI tools from someone who uses them well. Tests prove the function works. The checklist proves the function is maintainable.
When NOT to Use AI
Not every fix requires a prompt. Here are situations where typing the fix yourself is the right call:
Fix it manually when:
- The bug is a single word or operator (
andvsor,<vs<=) - You can see the fix immediately after reading the diff
- The change is a style issue (adding a type annotation, renaming a variable)
- The function is short (under 10 lines) and you understand every line
Re-prompt when:
- The fix requires restructuring a loop or changing the algorithm
- You can describe what is wrong but are not sure how to fix it
- Multiple lines need coordinated changes
Start over when:
- The function does something fundamentally different from what you asked
- Your prompt was missing so much context that patching would take longer than regenerating
The goal is efficiency: choose the strategy that gets you to correct, maintainable code in the fewest steps. Sometimes that means closing the AI chat and opening the file.
Phase 4 Boundary Note
This chapter gave you a process for iterating when you can read the error message and understand what went wrong. Sometimes you will encounter failures where the error message is not enough. The code runs without errors but produces the wrong result. Or the error message refers to a concept you have not learned yet. Or the failure happens only with certain inputs and you cannot figure out which ones.
When you cannot diagnose the problem from the error message alone, you need debugging tools: stepping through code line by line, inspecting variable values at each step, and isolating the exact point where behavior diverges from expectation. That is Phase 4.
PRIMM-AI+ Practice: The Full Cycle
Predict [AI-FREE]
Before starting the capstone, predict:
- At which funnel level will you write your first prompt? (1-5 confidence)
- How many rounds will you need to reach 6/6? (1-5 confidence)
- Will you use a manual fix at any point, or only re-prompts? (1-5 confidence)
Write these down before you begin.
Reflection prompts (check after completing the capstone)
After Round 1: How many tests passed? Was this more or fewer than you predicted? If fewer, which funnel level details were missing from your prompt?
After Round 2: Did a regression appear? If so, was it the type you expected (logic change during rewrite) or something different?
After Round 3: Did you use the 30% heuristic? Was the final fix manual or AI-generated? Was this the strategy you predicted?
Overall: How many rounds did it actually take? Compare to your prediction. If you overestimated, your prompt was better than you thought. If you underestimated, review the funnel levels and consider starting at a higher level next time.
Run
Complete the full capstone as described above. Fill in your tracking table at every round.
Investigate
After all tests pass, compare your final function to the Round 1 output. How many lines changed total? What percentage of the function was rewritten?
Modify
Take your final, working search_notes and add a new requirement: the function should also search within note.tags. Add a test for this:
def test_search_in_tags() -> None:
notes: list[Note] = [
Note(title="Cooking Tips", body="Use herbs.", tags=["recipes", "healthy"], word_count=2),
]
assert len(search_notes(notes, "healthy")) == 1
Apply the 30% heuristic: is adding tag searching a manual fix, a re-prompt, or a restart? Execute your chosen strategy.
Make [Mastery Gate]
Without any guidance, build a new function from scratch using the complete feedback loop:
def sort_notes_by_word_count(notes: list[Note], descending: bool = False) -> list[Note]:
"""Return a new list of notes sorted by word_count."""
...
Write at least four tests (ascending order, descending order, empty list, notes with equal word counts). Then prompt, iterate, and converge. Track your rounds. Your target: all tests passing in three rounds or fewer.
Try With AI
If Claude Code is not already running, open your terminal, navigate to your SmartNotes project folder, and type claude. If you need a refresher, Chapter 44 covers the setup.
Prompt 1: Process Review
I just completed a 3-round iteration on a search_notes function.
Here is my tracking table:
Round 1: 4/6 tests pass (case-insensitive and empty-term failed)
Round 2: 5/6 tests pass (regression on body search)
Round 3: 6/6 tests pass (manual fix: changed "and" to "or")
What could I have done differently in my original prompt to get
closer to 6/6 on the first try?
Read the AI's suggestions. Compare them to the funnel levels from Lesson 1. Did it recommend the same details you identified as missing?
Prompt 2: Self-Assessment
Here is my final search_notes function. Review it against these
criteria: type annotations on all variables, clear docstring,
no dead code, descriptive variable names, consistent style.
List anything that could be improved.
[paste your function here]
What you're learning: You are using the AI as a quality reviewer after the functional iteration is complete. This is the "verify before trust" step applied to code quality rather than test results.
Key Takeaways
-
The complete cycle is: prompt, test, diff, decide, repeat. Each step feeds the next. Tests tell you what is wrong, diffs tell you what changed, and the heuristic tells you what to do about it.
-
Verify before trust. All tests passing does not mean the code is done. Read it. Check types, docstrings, naming, and dead code. The quality checklist takes two minutes and catches issues that tests miss.
-
Know when NOT to use AI. Single-word fixes, style issues, and short functions are faster to edit manually. Save AI re-prompts for structural changes where you can describe the problem but not the solution.
-
When you cannot diagnose from the error message, you need debugging tools. That is Phase 4. This chapter gave you the process for iteration when you can read and understand the failures. Debugging handles the cases where you cannot.
Looking Ahead
You now have a repeatable process for collaborating with AI on code generation. You can write prompts, iterate through multiple rounds, read diffs, and make judgment calls about when to re-prompt, fix manually, or start over. The next chapter continues building your Python toolkit. When you encounter a problem you cannot diagnose from the error message alone, Phase 4 will give you the debugging tools to investigate deeper.