The Error Prediction
AI confident sound karti hai chahe woh right ho ya wrong. Jo student farq nahin pehchan sakta, woh AI ke saath us ke baghair se zyada dangerous hai.
Aap Chapter 1 ki Question Formulation skill use karenge taake apni error-detection queries design kar saken. Aapka seekha hua Reasoning Receipt format aage bhi chalega; is point ke baad AI output annotate karna second nature banne lagta hai.
Yeh chapter aap ko systematic error detector banne ki training deta hai. Vague skepticism ("AI par trust na karo") nahin, balki yeh precise, categorized analysis ke reasoning kahan aur kaise break hoti hai. Aap ek Error Taxonomy develop karenge jise aap baaki book mein har AI interaction par apply karenge.
Yeh Kyun Matter Karta Hai: James aur Authoritative Number
James Emma ke saamne chair par baitha aur laptop uski taraf saraka diya. Screen par supply chain disruption scenario ki three-paragraph AI analysis thi. Clean formatting. Numbered points. Specific percentages.
"Yeh parho," usne kaha. "Maine AI se wohi scenario analyze karwaya jo humne last week discuss kiya tha. Structure dekho. Data points dekho. Meri old job mein main jo bhi likhta, yeh us se behtar hai, aur is mein forty-five seconds lage."
Emma ne laptop touch kiye baghair screen parhi. "Kaun se parts correct hain?"
"Meri samajh ke mutabiq sab. Logic flow kar rahi hai. Numbers right lagte hain. Isne woh seasonal factor bhi catch kiya jo main apne prediction lock mein miss kar gaya tha."
"Tumne numbers verify kaise kiye?"
James ruk gaya. "Woh... specific hain. Response kehta hai supplier segment Q3 mein 12.4% contract hua. Yeh precise figure hai. 12.4% jaisa number koi bana tau nahin leta."
"Kyun nahin?"
"Kyun ke yeh oddly specific hai. Round numbers estimates jaise lagte hain. Decimal point wala number lagta hai jaise actual data se aya ho."
Emma aage jhuki. "Fabricated statistics ko dangerous banane wali cheez yehi hai. Precision research ka impression banati hai. Main tumhein keh sakti hun ke 73.6% first-year analysts AI-generated figures verify nahin karte, aur yeh 'most analysts check nahin karte' se zyada credible lagega. Lekin maine yeh number ten seconds pehle banaya hai."
James ne screen dobara dekhi. 12.4% figure wahan ab bhi utna hi solid lag raha tha jitna thirty seconds pehle. Farq sirf yeh tha ke ab woh nahin bata sakta tha ke yeh real hai ya sirf real sound karta hai.
"Ruko." Usne response scroll kiya. "Tau tum keh rahi ho in mein se koi bhi number fabricated ho sakta hai? Puri cheez consulting report jaisi lagti hai. Kya tum mujhe keh rahi ho ke jo bhi professional sound kare us par distrust karun?"
"Main tumse different question pooch rahi hun. Jab tumne yeh response parha, tumne ise evaluate karne ke liye kya use kiya?"
James ne socha. "Jis tarah yeh likha gaya tha. Confidence. Structure."
"Content nahin. Presentation." Emma ne is distinction ko baithne diya. "Tumne report ko us tarah evaluate kiya jaise PowerPoint deck evaluate karte ho. Kya yeh polished lagti hai? Kya flow karti hai? Kya speaker sure lagta hai? Meri first operations role mein hum ise 'pitch-deck syndrome' kehte the. Best-formatted proposal contract jeet leti thi, chahe numbers hold up karte ya nahin."
"Main yeh nahin kar raha tha."
"Tumne mujhe kaha numbers 'right lagte hain' kyun ke un mein decimal points hain. Tum exactly yehi kar rahe the."
James peechay ho kar baith gaya. Woh sahi thi, aur woh mehsoos kar raha tha ke ise admit karne mein resistance hai. Screen par analysis ab bhi convincing lag rahi thi. Problem yehi thi. Woh nahin bata sakta tha ke woh convincing is liye lag rahi thi kyun ke correct thi, ya is liye ke well-written thi. Aur five minutes pehle tak use pata hi nahin tha ke in dono mein farq hai.
"Alright," usne kaha. "Tau alternative kya hai? Main har sentence fact-check nahin kar sakta. Us mein analysis khud likhne se zyada waqt lagega."
"Tum har sentence fact-check nahin karte. Tum seekhte ho errors kahan chhupti hain. Patterns hote hain. Specific, predictable patterns. Error Taxonomy isi ke liye hai."
Emma khari hui aur apni coffee uthai. "Exercise start karne se pehle main chahti hun tum ek kaam karo. Is AI response ko dekho aur predict karo ke errors kahan hain. Apni predictions likho. Unhein categorize karo. Phir exercise run karo aur dekho tumhari predictions actual findings se kaise compare hoti hain."
Woh darwaze par ruki. "Prediction detection se zyada matter karti hai. Jab koi bata de ke kahan dekhna hai tau errors koi bhi dhoond leta hai. Skill yeh hai ke kisi ke batane se pehle pata ho kahan dekhna hai."
She left.
James screen par analysis ko dekhta raha. Teen paragraphs pehle yeh finished product lagti thi. Ab yeh ek aisa exam lag rahi thi jiske liye usne study nahin ki thi.
Error Taxonomy
| Category | Is Ka Matlab |
|---|---|
| Factual error | Aisa claim jo demonstrably false ho |
| Logical gap | Aisa conclusion jo premises se follow nahin karta |
| False confidence | Uncertain information ko unjustified certainty ke saath state karna |
| Missing context | Aise crucial factors omit karna jo analysis change kar dete |
| Correlation-causation confusion | Correlation ko causation ka proof samajhna |
| Outdated information | Aisa data ya facts use karna jo ab current nahin hain |
| Fabricated citation | Aise source ka reference dena jo exist nahin karta |
| Cultural blind spot | Yeh assume karna ke ek cultural context universally apply hota hai |
Exercise 1: The Error Prediction
Layers Used: Layer 1 (Predict Before You Prompt), Layer 2 (Reasoning Receipt)
James ek AI response ko dekh raha hai jis par ab woh face value par trust nahin kar sakta. Aap bhi.
Aap ne Chapter 1, Exercise 1 mein Prediction Lock format use kiya tha. Wahan aap ne question quality predict ki thi; ab aap error types predict karenge.
Apni Error Prediction Likhein (AI touch karne se pehle)
Step 1. Apni sealed prediction likhein (~15 min, no AI). Kisi bhi AI ko prompt karne se pehle, likhein:
- Aapke khayal mein correct analysis mein kya shamil hona chahiye (key factors, tradeoffs, data needed)
- Aap kahan predict karte hain ke AI apni analysis mein strong hogi
- Aap kahan predict karte hain ke AI errors karegi ya important context miss karegi: start karne ke liye teen categories par focus karein: factual error, false confidence, aur missing context. Baqi five categories reference ke liye table mein hain, lekin yeh teen spot karna sab se asan hain
Yeh aapka prediction document hai. Step 2 par jane se pehle ise likhein.
Step 2. Get two AI responses (~10 min). Ek choose karein scenario below, phir prompt two different AI tools ke saath identical question. Dono full responses save karein.
Annotate Aur Compare Karein
Step 3. Key claims annotate karein (~20 min). Har AI response ko parhein. Dono responses mein se 5 strongest claims aur 5 most suspicious claims choose karein, total 10. Har claim ko upar wali Error Taxonomy se label karein. Agar claim correct hai tau use "no error detected" mark karein.
Step 4. Apni comparison table banayein (~10 min). Step 1 ki predictions ko Step 3 mein milne wale actual errors se compare karein (template neeche hai). Count karein ke dono tools mein har error type kitni dafa mili.
Apna Scenario Choose Karein
- Policy
- Technical
- Education
Scenario A (Policy): "Should developing nations invest heavily in growing power demands meet karne ke liye nuclear energy mein heavily invest karna chahiye? "
Scenario B (Technical): "Should companies migrate their entire infrastructure ko serverless architecture par migrate karna chahiye? "
Scenario C (Education): "Should universities replace traditional lectures with AI-powered personalized tutoring?"
Ek choose karein.
- Aapka sealed prediction document (AI se pehle written) jismein expected strengths aur error types listed hon
- Do AI responses jahan aap ki 10 key claims Error Taxonomy use karte hue annotated hon
- A comparison table: predicted errors vs. actual errors jo mile (see template below)
- A count ka each error type found across both tools
Prediction Document Template (click to expand)
PREDICTION DOCUMENT (AI ko prompt karne se pehle yeh likhein)
Chosen scenario: ___
Correct analysis mein kya cover hona chahiye:
- Key factor 1: ___
- Key factor 2: ___
- Key factor 3: ___
Jahan main predict karta hun ke AI strong hogi:
Jahan main predict karta hun ke AI errors karegi:
| Predicted Error Type (from taxonomy) | Mujhe yeh error kyun expect hai |
|---|---|
| e.g. Missing context | AI won't know about recent policy changes mein [country] |
| e.g. Cultural blind spot | AI is global issue ke liye Western/US context assume karegi |
Prediction vs. Reality Comparison Table (click to expand)
| Predicted Error | Kya yeh hua? | Actual Error Jo Mila (if different) | Error Category |
|---|---|---|---|
| Yes / No |
Error Count Summary:
| Error Category | AI tool 1 | AI tool 2 | Total |
|---|---|---|---|
| Factual error | |||
| Logical gap | |||
| False confidence | |||
| Missing context | |||
| Correlation-causation confusion | |||
| outdated information | |||
| Fabricated citation | |||
| Cultural blind spot |
Main AI-generated analysis mein errors detect karna seekh raha hun. Maine ek scenario question ke bare mein do different AI tools se poocha aur phir Error Taxonomy (factual error, logical gap, false confidence, missing context, correlation-causation confusion, outdated information, fabricated citation, cultural blind spot) use karte hue dono responses annotate kiye. Please:
(1) Meri error annotations review karein -- kya maine har error correctly identify kiya? False positives flag karein (jo cheezen maine error mark ki hain lekin actually correct hain) aur false negatives bhi (jo errors maine miss kiye). (2) Meri error detection accuracy percentage mein rate karein. (3) Har missed error ke liye explain karein ke mujhe use kaise catch karna chahiye tha. (4) Error Taxonomy ke mere use ko rate karein -- kya main errors correctly categorize kar raha hun ya misclassify kar raha hun? (5) Meri error detection mein kaun se patterns dikhte hain -- main kaun si types catch karne mein acha hun aur kaun si consistently miss karta hun?
Yeh AI responses meri annotations ke saath hain:
Yeh meri prediction document hai:
Aakhir mein, is exercise ke liye Thinking Score Card complete karein: Independent Thinking (1-10), Critical Evaluation (1-10), Reasoning Depth (1-10), Originality (1-10), Self-Awareness (1-10). Har score ke liye one-sentence justification dein.
Discuss with an AI. Question your scores.
Come back when you have your BEST evaluation.
James Ke Saath Kya Hua
James ne apna prediction document aur annotated AI responses side by side rakh diye. Usne predict kiya tha ke AI structure aur logical flow mein strong hogi. Yeh correct tha. Usne specific statistics mein factual errors predict kiye the. Use do mile, lekin teen aise bhi mile jo usne bilkul predict nahin kiye the: false confidence ke do cases aur ek missing-context error jisne poora conclusion change kar diya.
False confidence errors ne use sab se zyada disturb kiya. Dono dafa AI ne contested reality ko settled fact ki tarah state kiya tha. Aur dono dafa authoritative tone ne use almost convince kar diya tha ke woh unhein skip kar de.
"Maine total five errors catch kiye," usne Emma ke wapas aane par kaha. "Lekin maine un mein se sirf two predict kiye the. Baqi three categories tau maine consider hi nahin ki thin."
"Kaun si category ne tumhein sab se zyada surprise kiya?"
"False confidence. AI ne nahin kaha 'this might be the case.' Isne kaha 'this is the case.' Aur maine almost accept kar liya kyun ke sentence well-constructed tha." Woh ruka. "Yeh meri old company ke quarterly reviews jaisa hai. Jo managers sab se zyada certainty ke saath bolte the unhein sab se kam pushback milta tha, even jab unke numbers baaki sab se weaker hote the."
"Ab tumhare paas us pattern ke liye vocabulary hai. Aur use check karne ka reflex bhi."
Jo Lesson Seekha Gaya
Error detection specific categories wali trainable skill hai, sirf yeh vague feeling nahin ke kuch off hai. AI errors dekhne se pehle unhein predict karke, aap yeh internal model banate hain ke AI kahan fail hoti hai. Aapki predictions aur actual errors ke darmiyan gap exactly batata hai ke aap ki instincts kaun se failure modes miss karti hain. Aapki next improvement wahi gap hai.