The Contradiction Test
Why This Matters: James and the False Consensus
You will use the Reasoning Receipt format from Chapter 1, Exercise 1. Your annotation practice from Chapter 1 becomes error annotation here.
James set two AI responses side by side on his screen, both answering the same question. He pointed at the screen. "These two agree on almost everything. Both say remote work improves individual productivity. Both cite flexible scheduling. Both mention reduced commute time. If two separate tools land on the same answer independently, that's a pretty strong signal, right?"
"Is it?"
"In my old job, when two independent auditors gave the same finding, we treated it as confirmed. That's how verification works. Multiple sources, same conclusion."
Emma pulled her chair closer to the screen. "Read me the third paragraph from each response."
James read them. The first tool argued that remote teams collaborate just as effectively through asynchronous tools. The second argued that in-person interaction produces measurably more creative output. He stopped.
"Wait. Those are saying opposite things."
"Keep reading. Paragraph five."
James scrolled. One cited cost savings of 30% on office overhead. The other cited studies showing companies that returned to offices saw higher employee engagement. He sat back.
"Okay, so they agree on some things and contradict each other on others. But the parts where they agree are still reliable, right?"
"Why?"
"Because... two tools reaching the same conclusion means..." He trailed off. "Okay, let me think about this. They're both trained on similar data. So when they agree, it might not be two independent conclusions. It might be the same conclusion from the same source, repeated twice."
"Now you're asking the right question. Agreement between AI tools is not the same as independent verification. It could mean they're both right. It could also mean they absorbed the same flawed source."
James looked at the two responses. A minute ago, the areas of agreement had felt like solid ground. Now he wasn't sure any of it was solid.
"So what do I do with the contradictions? Pick whichever one sounds better?"
"You build something better than both. That's the exercise."
Exercise 2: The Contradiction Test
Layers Used: Layer 4 (Contradiction Challenge), Layer 6 (Iterative Drafts)
James is looking at two confident answers that cannot both be right. So are you.
Map the Divergence Points
Step 1. Ask two AI tools the same question (~5 min). Choose a nuanced question where reasonable people disagree; for example: "Is remote work better for productivity than office work?" Prompt two different AI tools with the identical question. Save both full responses.
Step 2. Identify divergence points (~15 min). Read both responses side by side. For every point where the two tools disagree, write a divergence annotation (see example below). For each divergence, determine: which side has stronger evidence, and which is asserting without support.
Build Your Third Analysis
Step 3. Write your Draft 1 analysis (~20 min). Write your own third analysis (500-800 words) that is more rigorous than either AI output. Use the divergence points as your starting material. You should do better than both tools on the contested points.
Step 4. Get AI critique (~5 min). Submit your Draft 1 to the AICheck below. The AI will critique your analysis and identify weaknesses.
Iterate Through Three Drafts
Step 5. Revise to Draft 2 (~15 min). Each draft should be genuinely better, not just edited for grammar. Based on the AI critique, focus on your weakest claims: strengthen the evidence, remove unsupported assertions, or acknowledge uncertainty. Write an evolution note explaining what you changed and why.
Step 6. Final reflection to Draft 3 (~15 min). Re-read your Draft 2 with fresh eyes. Ask yourself: where am I still asserting without evidence? What would someone who disagrees attack first? Revise those sections. Write a second evolution note. This is your final submission.
- The two AI responses with divergence points annotated (see example below)
- Your Draft 1 third analysis (before AI feedback)
- Your Draft 2 (revised after AI critique) + evolution note
- Your Draft 3 (final) + evolution note
Divergence Annotation Example (click to expand)
How to write a divergence annotation:
Divergence #1: Remote work and collaboration
- AI Tool 1 claims: Remote teams are equally productive because async communication is more efficient
- AI Tool 2 claims: In-person collaboration produces 15% more creative output due to spontaneous interactions
- Evidence assessment: Tool 2 cites a specific study (check if it exists). Tool 1 makes a general assertion without evidence.
- Verdict: Tool 2 has stronger support on this point, but the cited study needs verification.
Do this for every meaningful disagreement between the two responses.
Evolution Note Example (click to expand)
Strong evolution note (Draft 1 → Draft 2): "The AI critique pointed out that my analysis assumed all remote work is the same, ignoring the difference between fully remote and hybrid. I restructured Section 2 to separate these cases. I also removed my claim about cost savings because I had no evidence for the specific figure I used."
Weak evolution note: "I fixed some wording and added more detail."
Each note should point to specific changes and why you made them.
I am learning to detect contradictions between AI outputs and build a more rigorous analysis through iterative drafts. I asked two AI tools the same question and received contradictory responses. I identified divergence points, then wrote three progressively improved drafts of my own analysis. Please:
(1) Evaluate my divergence annotations -- did I catch all the meaningful contradictions? Did I correctly assess which side had stronger evidence? (2) Grade my Draft 1 on a scale of 1-10 for rigor, originality, and evidence quality compared to the two AI responses. (3) Identify the 3 weakest claims in Draft 1 and explain what would make them stronger. (This is your critique for me to use in Draft 2.) (4) Evaluate my Draft 2 and Draft 3 evolution -- did I make substantive improvements or just cosmetic changes? Rate the evolution quality. (5) Grade my final Draft 3 on a scale of 1-10. What did both AI tools get wrong that I also missed, even in my final draft?
Question:
AI Response 1:
AI Response 2:
My divergence annotations:
My Draft 1 analysis:
My Draft 2 analysis + evolution note:
My Draft 3 (final) + evolution note:
Finally, complete the Thinking Score Card for this exercise: Independent Thinking (1-10), Critical Evaluation (1-10), Reasoning Depth (1-10), Originality (1-10), Self-Awareness (1-10). For each score, give a one-sentence justification.
Discuss with an AI. Question your scores.
Come back when you have your BEST evaluation.
What Happened With James
James flipped between his three drafts on screen. Draft 1 had been a patchwork: the strongest-sounding claims from each AI tool stitched together with his own transitions. Draft 2, after the AI critique, had lost two paragraphs entirely because he realized they were assertions without any evidence behind them. Draft 3 was shorter than either AI response, but he could defend every sentence in it.
"The weird thing," he said, "is that Draft 3 is the one I'm most confident about, and it's the one with the most qualifications. 'The evidence suggests' instead of 'this is true.' 'Under certain conditions' instead of blanket statements."
"You sound less certain."
"I am less certain. But I know exactly what I'm uncertain about and why. Draft 1 was confident about everything because I hadn't examined anything." He thought for a moment. "It's like those project status reports at my old company. The ones that said 'everything is on track' were the most dangerous. The ones that said 'we have three risks and here's our mitigation plan' were the ones you could actually trust."
"Confidence without examination is just noise. You built something quieter. And stronger."
The Lesson Learned
When two confident AI responses disagree, the disagreement is a signal to think harder, not a reason to pick one. Building a third analysis that improves on both forces you to evaluate evidence instead of inheriting conclusions. The three-draft evolution reveals whether you can integrate feedback and genuinely improve, or whether you stop thinking after the first attempt.