Chapter Summary

This chapter began with a problem: the knowledge that makes a domain agent genuinely useful is tacit — it resists articulation — and no platform, model improvement, or prompt engineering technique solves this without a structured extraction methodology. It ends with the complete methodology. The nine lessons between those two points did not add complexity for its own sake. Each one answered a question that the previous lesson raised.

The articulation gap raised the question of how to surface tacit knowledge. The five-question interview framework answered it for knowledge that lives in expert heads. The interview protocol made the framework operational. The document extraction framework answered it for knowledge that lives in institutional documents. The domain-method mapping determined when to use each method and how to reconcile them when both apply. The SKILL.md translation lesson turned extraction outputs into structured instructions. The validation scenario set tested whether those instructions actually work. The validation loop turned test failures into targeted improvements. And the hands-on exercise proved that the methodology produces results when applied to a real domain.

That chain is the chapter. Understanding it as a chain — not as nine separate lessons — is the synthesis this summary is for.

The Methodology in Sequence

Each lesson answered a specific question. Each answer led directly to the next question.

Lesson	Question Answered	Key Output
L01: The Problem That No Platform Solves	Why does a domain agent need structured extraction?	Tacit vs explicit knowledge; the articulation gap; why platforms cannot solve it
L02: The Five Questions	How do you surface tacit knowledge from an expert's head?	Five interview questions targeting decision logic, exceptions, and escalation
L03: Conducting the Expert Interview	How do you run the interview so the expert articulates rather than performs?	Briefing protocol; specific note-taking; north star summary
L04: The Document Extraction Framework	How do you extract knowledge from institutional documents?	Three-pass framework: explicit rules, contradiction mapping, gap identification
L05: Choosing and Combining Methods	Which method do you use for a given domain?	Domain-method mapping; reconciliation principle for A+B domains
L06: From Extraction to SKILL.md	How do you translate extraction outputs into structured instructions?	Persona (Chapter 15's four structural elements addressed through three extraction-focused writing questions), Questions (capability + out-of-scope), Principles (testable)
L07: Building the Validation Scenario Set	How do you test whether the instructions actually work?	Four scenario categories at defined proportions; three scoring components; 95% threshold
L08: The Validation Loop	How do you turn test failures into improvements?	Failure pattern interpretation; targeted rewriting; shadow mode; graduated autonomy
L09: Hands-On Exercise	Can I do this?	Complete extraction-to-validation cycle for a real domain

Three Insights That Connect the Methodology

Reading the nine lessons as a sequence reveals three insights that no individual lesson states on its own.

The first is that extraction and validation are interdependent, not sequential add-ons. An extraction that produces a SKILL.md which is never validated against scenarios is a document that encodes the expert's knowledge but has unknown coverage gaps — it handles the cases the expert thought to mention and fails on the ones they did not. A validation that tests a SKILL.md which was not produced through structured extraction is a test of instructions that do not contain the expert's actual decision-making logic — the scenarios may pass, but the agent is not doing what the expert would do. The methodology requires both stages because each one catches the failures the other cannot.

The second insight is that the credit analyst example was not an illustration — it was a proof of concept. Every lesson applied the same methodology to the same domain, and by Lesson 9, the cumulative result was a testable SKILL.md draft. The methodology is not theoretical. It produces a concrete artifact that can be validated, revised, and deployed.

The third insight is that the methodology is domain-agnostic but the application is domain-specific. The five interview questions work for any domain. The three-pass document extraction works for any document corpus. The four validation categories apply to any agent. But the specific questions you ask, the specific documents you extract from, the specific scenarios you design, and the specific scoring criteria you apply all depend on the professional context. This is why the domain chapters that follow are necessary — not to teach a different methodology, but to show how this one works when applied to finance, legal, HR, healthcare, architecture, sales, and operations.

Self-Assessment Checklist

Before continuing, verify that you can answer these questions with specificity. Generic answers indicate a concept that needs review.

• Tacit vs explicit knowledge: Can you define both types, provide a domain-specific example of each, and explain why tacit knowledge is the one that determines whether the agent is genuinely useful?
• Method A: Can you state the five interview questions, explain what each one surfaces, and describe the briefing protocol that sets the collaborative mode?
• Method B: Can you describe the three passes, name the three types of contradictions, and explain the two response patterns for gaps?
• Method selection: Can you classify your domain as A-primary, B-primary, or A+B, and state the reconciliation principle for when expert judgement and documented standards conflict?
• SKILL.md translation: Can you write a Persona that addresses Chapter 15's four structural elements through three extraction-focused writing questions, a Questions section with explicit out-of-scope boundaries, and Principles that are specific enough to test?
• Validation scenarios: Can you design a twenty-scenario set with the correct proportions and score each output on accuracy, calibration, and boundary compliance?
• Shadow mode: Can you state the entry criteria and explain why scenario testing and shadow mode validate different things?

If any of these are uncertain, revisit the relevant lesson before continuing. The domain chapters that follow assume the methodology is understood and proceed directly to domain-specific application.

Methodology Quick Reference

This checklist consolidates the full Knowledge Extraction Method into a single reference you can use during your first real extraction. Each step references the lesson where the technique is taught in full.

1. Classify your domain (L05)

• Method A-primary (knowledge in expert heads): finance, sales
• Method B-primary (knowledge in documents): HR, operations
• A+B (knowledge in both): legal, healthcare, architecture

2. Extract — Method A (L02, L03)

• Brief the expert: purpose, output, process expectation
• Ask the five questions with follow-ups:
  • Q1: Recent success (decision-making logic)
  • Q2: Instructive failure (defensive knowledge)
  • Q3: Junior vs senior gap (expertise differential)
  • Q4: One-page decision guide (load-bearing heuristics)
  • Q5: Automation boundaries (escalation conditions)
• Capture specific statements, not generic ones
• Write the north star summary immediately: paragraph 1 = decision-making logic, paragraph 2 = escalation condition

3. Extract — Method B (L04)

• Pass One: extract every explicit rule as "The agent should [X] when [Y]"
• Pass Two: map contradictions (temporal, jurisdictional, interpretive)
• Pass Three: identify gaps (low-stakes = apply-and-flag; high-stakes = escalate)

4. Reconcile if A+B (L05)

• Documented standards take precedence for regulatory compliance
• Expert judgement takes precedence for operational decisions within professional competence

5. Write the SKILL.md (L06)

• Persona: professional level, quality standards, uncertainty behaviour
• Questions: in-scope categories + out-of-scope boundaries (equally precise)
• Principles: specific, testable instructions with uncertainty calibration vocabulary

6. Validate (L07, L08)

• Build scenario set: 50% standard, 25% edge, 15% adversarial, 10% high-stakes (minimum 20)
• Score each output: accuracy, calibration, boundary compliance (all three must pass)
• Threshold: 95% overall + zero high-stakes failures
• Interpret failure patterns → targeted rewrite of two weakest instructions → re-test
• Shadow mode: 30 days minimum, production inputs, same scoring rubric
• Graduated autonomy: standard cases first, extend based on performance record

What Comes Next

The methodology does not change. The five interview questions, the three-pass document extraction, the reconciliation principle, the SKILL.md structure, and the validation loop are the permanent toolkit. What changes in each domain chapter is the context in which they are applied.

Chapter 17 opens with finance — the domain the credit analyst example has been preparing you for. Where this chapter used the credit analyst to teach the methodology, Chapter 17 uses the methodology to build a production-grade financial analysis agent. The extraction targets shift from general credit analysis to specific financial workflows. The validation scenarios shift from illustrative examples to domain-specific test cases grounded in real financial instruments and regulatory requirements. The shadow mode criteria shift from generic thresholds to metrics that a compliance function would accept.

The architecture from Chapter 15 does not change. The methodology from Chapter 16 does not change. What the domain chapters add is the professional knowledge that makes the methodology produce a SKILL.md worth deploying.

Try With AI

Use these prompts in Anthropic Cowork or your preferred AI assistant to integrate the chapter's methodology.

Prompt 1: Personal Methodology Mapping

I have just completed Chapter 16 on the Knowledge Extraction Method.
I work as [YOUR ROLE] in [YOUR INDUSTRY]. Help me map the full
methodology to a specific domain agent I want to build:
[DESCRIBE THE AGENT IN 2-3 SENTENCES].

Walk me through each stage:
1. Problem: What tacit knowledge in this role would the agent need?
2. Method selection: Is this domain A-primary, B-primary, or A+B?
3. Extraction plan: What would the five interview questions surface?
   What documents would Method B target?
4. SKILL.md outline: What would the Persona, Questions, and Principles
   sections need to address?
5. Validation: What would a standard, edge, adversarial, and high-stakes
   scenario look like for this agent?

Identify any gaps where I would need information I do not currently
have to complete a stage.

What you're learning: How to apply the complete chapter methodology to a real domain agent. This synthesis exercise forces you to use every stage — problem identification, method selection, extraction, translation, and validation — in sequence for a specific use case, revealing which parts of the methodology you have understood deeply and which remain abstract.

Prompt 2: Methodology Comparison

Compare applying the Knowledge Extraction Method to two different
domains:
(1) A senior credit analyst at an investment bank, where the critical
knowledge is pattern recognition in financial statements and the
instinct for which numbers to question.
(2) An HR compliance specialist at a multinational, where the critical
knowledge is in policy handbooks, employment law across jurisdictions,
and documented grievance procedures.

For each domain:
- Which extraction method is primary and why?
- What would the five interview questions surface that documents cannot?
- What would the three-pass extraction surface that the interview cannot?
- Where would expert judgement and documented standards most likely
  conflict?
- What would a high-stakes validation scenario look like?

Explain why the same methodology produces different extraction plans
for these two domains.

What you're learning: How the methodology adapts to domain context. The extraction framework is consistent across domains — the same five questions, the same three passes, the same four validation categories — but the specific knowledge surfaced, the method priority, and the reconciliation decisions differ fundamentally based on where the critical knowledge lives. Comparing two contrasting domains makes this adaptation concrete rather than theoretical.

Prompt 3: Gap Analysis

I am about to start the domain chapters (Finance, Legal, HR,
Healthcare, Architecture, Sales, Operations). Before I do, help me
identify which parts of the Knowledge Extraction Method I need to
review.

Ask me five diagnostic questions — one for each major stage of the
methodology (extraction methods, interview technique, document
extraction, SKILL.md translation, and validation). For each question,
assess whether my answer demonstrates operational understanding (I
could do this tomorrow) or conceptual understanding (I know what it
is but would need to review before doing it).

Based on my answers, recommend which lessons I should revisit before
starting the domain chapters.

What you're learning: The gap between conceptual understanding and operational readiness is the gap that determines whether the domain chapters will be productive or frustrating. This prompt simulates a readiness assessment — surfacing the stages where you know the concept but would struggle with the execution, so you can review those lessons before applying the methodology to a specific domain.

Flashcards Study Aid

---

Continue to Chapter Quiz →