Append-Only Events: Audit Trail That Survives the Session

It is Wednesday morning. Finance opens a ticket.

"What did the Reconciler do for Alice on Tuesday between 11:30 and midnight? We need it for the auditor by lunch."

You open the database. You see Alice's current expenses table. You see her runs row, her turns, her tool_calls from Lesson 2. You can answer some of it. You cannot answer all of it.

You see the rows the Reconciler wrote. You do not see what it considered and rejected. You do not see which transactions it flagged as low confidence. You do not see the email intent that was queued, retried, and finally delivered. The current state is the snapshot. The events are the movie.

You only have the snapshot.

This lesson teaches you to record the movie.

Key Terms for This Lesson

• Event: A single immutable row that says "this thing happened, at this time, with this payload."
• Append-only: A table that accepts INSERT but not UPDATE or DELETE. Past rows cannot be rewritten.
• Immutable history: The full ordered sequence of events for a subject. Once written, it stays.
• Replay: Reconstructing what happened by reading events in order, even if the snapshot has been overwritten.
• No event, no audit: If the Worker did not emit an event for a transition, the auditor cannot see it. Silence is loss.

State vs Events

A database has two different jobs once a Worker is the writer. Most teams only do one of them.

Question	Answered by	Example
What is true now?	State	"Alice has 47 expenses totaling $1,820 this month."
What happened, in what order, and why?	Events	"At 23:43 the Reconciler claimed transaction T-91, categorised it as Food at 0.91 confidence, wrote one expense, queued one email."

State is the snapshot finance reads in dashboards. Events are what auditors and replays demand. Both are needed. State alone cannot answer "what happened on Tuesday." Events alone cannot answer "what is the balance right now." A system of record carries both.

What an agent_events Table Looks Like

CREATE TABLE agent_events (
    id           BIGSERIAL PRIMARY KEY,
    occurred_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
    run_id       UUID NOT NULL,
    turn_seq     INT,
    event_type   TEXT NOT NULL,
    subject_type TEXT,
    subject_id   TEXT,
    payload      JSONB NOT NULL
);

Six things to read.

1. id BIGSERIAL: a monotonic sequence so events have a stable ordering even when timestamps tie.
2. occurred_at: when the event happened, in the database's clock.
3. run_id and turn_seq: the run and turn from Lesson 2's journal. Every event is anchored to a run.
4. event_type: a short string like claimed_transaction or expense_written.
5. subject_type and subject_id: the thing the event is about (a pending_transaction row, an expense row).
6. payload JSONB: the body of the event. The fields the auditor will need.

In normal operation, this table only sees INSERTs. No UPDATE. No DELETE. The next section is how you make sure that stays true.

How to Make It Append-Only

The table itself does not enforce append-only. You enforce it with the role.

GRANT INSERT, SELECT ON agent_events TO worker_write;
REVOKE UPDATE, DELETE ON agent_events FROM worker_write;

Read in plain English: "The Worker role can write new event rows and read existing ones. It cannot change them and it cannot remove them."

Some teams add a trigger as a second wall.

CREATE OR REPLACE FUNCTION reject_event_mutation()
RETURNS TRIGGER AS $$
BEGIN
    RAISE EXCEPTION 'agent_events is append-only';
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER block_event_update BEFORE UPDATE OR DELETE ON agent_events
FOR EACH ROW EXECUTE FUNCTION reject_event_mutation();

You do not need to write the trigger. You need to recognise it when the agent produces it. The trigger is a belt; the role is the suspenders. Together they make the table behave like a logbook.

Three Reconciler Events Worth Recording

For the Expense Reconciler, four event types carry most of the audit value.

event_type	When emitted	Sample payload fields
claimed_transaction	After a successful FOR UPDATE SKIP LOCKED claim (Lesson 4)	pending_transaction_id, user_id
categorised_with_confidence	After the model returns a category and a confidence score	category, confidence, model_version
expense_written	After the idempotent INSERT into expenses (Lesson 3)	expense_id, amount, idempotency_key
email_dispatched	After the outbox row is marked delivered (Lesson 5)	outbox_id, recipient, delivered_at

Example payload for categorised_with_confidence:

{
    "category": "Food",
    "confidence": 0.91,
    "model_version": "deepseek-v4-flash",
    "input_hash": "sha256:f1a2..."
}

The payload is a JSON document, not a relational row. It records the decision, not the rules around it. Auditors read it like a diary entry.

Replay From Events (the Small Version)

Given the events for one run_id, ordered by id, you can reconstruct what the Worker did, even if the expenses table has been edited since.

SELECT occurred_at, event_type, payload
FROM agent_events
WHERE run_id = 'a3f1...'
ORDER BY id;

Six rows, in order, become a paragraph: at 23:43 the Reconciler claimed transaction T-91 for Alice; categorised it as Food at 0.91 confidence; wrote expense E-2231; queued email O-405; the email worker delivered O-405 at 23:44; the run completed.

You did not ask the model what happened. You read the rows.

PRIMM-AI+ Practice: Reconstruct Tuesday Night

Predict [AI-FREE]

You are given these six event rows for a single run. Before reading anything else, write the Reconciler's actions in plain English in the order they occurred.

occurred_at	event_type	subject_type	subject_id	payload (excerpt)
23:43:01	claimed_transaction	pending_transaction	T-91	{ "user_id": "alice" }
23:43:02	categorised_with_confidence	pending_transaction	T-91	{ "category": "Food", "confidence": 0.91 }
23:43:03	expense_written	expense	E-2231	{ "amount": 18.40, "idempotency_key": "..." }
23:43:03	email_dispatched	outbox_message	O-405	{ "queued": true }
23:43:59	email_dispatched	outbox_message	O-405	{ "delivered_at": "23:43:59" }
23:44:00	run_completed	run	R-44	{ "expense_count": 1 }

Write down:

• A one-paragraph narrative of what happened.
• Whether the snapshot (the expenses row) is enough to reconstruct this without the events.
• Your confidence score from 1 to 5.

Run

Ask Claude Code to insert the six event rows into a sample agent_events table on a Neon branch, then run SELECT ... ORDER BY id to read them back.

You should see the six rows in the same order you predicted. The narrative should match.

Investigate

Write your own explanation:

1. The events are ordered by id, not just by timestamp, so two events at the same second still have a stable order.
2. The snapshot in expenses shows expense E-2231 exists. It does not show that the categorisation confidence was 0.91, or that the email was queued before it was delivered.
3. Removing the categorised_with_confidence row would still leave a valid expense, but the auditor could not check whether the model was confident or guessing.
4. Removing the second email_dispatched row would make it look like the email was queued and never delivered.

Then ask the agent:

1. "If the categorised_with_confidence event is missing, what auditor question becomes unanswerable?"
2. "What event would I need to add to record a rejected transaction (one the Reconciler decided not to categorise)? Sketch the payload."
3. "Why does ordering by id matter more than ordering by occurred_at when two events share a timestamp?"

Modify

Try to UPDATE one of the event rows.

UPDATE agent_events
SET payload = jsonb_set(payload, '{confidence}', '0.99'::jsonb)
WHERE id = 2;

Predict the outcome before running.

• If the role is worker_write and UPDATE was revoked, the database refuses with a permission error.
• If the role is worker_admin (or no revoke is in place), the UPDATE succeeds and silently rewrites history. The audit trail is now compromised and nothing in the table tells you that.

The lesson: the role grants are the contract. Without them, the events table is a logbook with a pencil and an eraser.

Make [Mastery Gate]

Write a one-sentence brief: the Reconciler should also emit a low_confidence_flag event whenever the categorisation confidence is below 0.6, with the threshold and the actual score in the payload.

Hand the brief to the agent. Ask for:

1. The event_type string.
2. The payload shape (named fields, not free-form).
3. The exact moment in the Reconciler's flow where the event is inserted.

Read the response. The gate passes when you can name what the event lets the auditor answer that the snapshot cannot, and when the payload includes both the threshold and the score (not just one of them).

Try With AI

Prompt 1: Write Your Own Event Taxonomy

Help me list the events the Expense Reconciler must emit so an auditor
can fully replay a run without looking at the snapshot. For each event,
give me the event_type string and the three to five payload fields the
auditor would care about. Do not propose more than seven event types.

What you're learning: Event design is a discipline of subtraction. Too few events and you cannot replay. Too many and the table becomes noise. Seven is a useful budget for a single Worker.

Prompt 2: Replay Drill

I will paste eight event rows for a single run_id. Read them in order
and write a paragraph in plain English describing what the Worker did.
Do not invent events that are not in the data. If something is missing,
say what cannot be reconstructed.

What you're learning: Replay is a reading skill, not a writing skill. The events name what happened; your job is to narrate, in order, without filling in gaps.

Prompt 3: Append-Only Enforcement Check

Look at this DDL for an agent_events table. Tell me whether the table
is actually append-only or only nominally append-only. List every way
a Worker with default privileges could still corrupt the audit trail.
Then propose the smallest set of GRANT/REVOKE statements that close the
gaps.

What you're learning: A table is append-only only when the role cannot UPDATE or DELETE it. Reading the DDL is half the check. Reading the GRANTs is the other half.

Checkpoint

• I can explain the difference between state ("what is true now?") and events ("what happened, in what order?").
• I can read a sequence of agent_events rows for one run_id and narrate the run in plain English.
• I can recognise when an events table is append-only in name only because UPDATE and DELETE were not revoked.
• I know which Reconciler events I would refuse to ship without (claimed, categorised, written, dispatched).
• I can name at least one auditor question the snapshot cannot answer that the events can.

Flashcards Study Aid