by.waclaw.online / pm-agent / 04

The Librarian: Keeping the Catalog Alive

Part 4 of 8 — the hardest promise in the guide: that the catalog is current. This is how the Librarian keeps it, by treating documentation exactly like a codebase.

The promise, and why it is hard

A catalog is only worth anything if you can believe it. The moment a reader has to ask "but is this still true?" the documentation has already failed — it is back to being the rotting wiki from chapter 1, technically present and practically useless. The entire value of the system rests on one promise: the catalog tracks reality. This chapter is how the Librarian keeps that promise.

The Librarian's stance is simple and unglamorous: docs are code. Every entity has an owner. Every claim has provenance. Every change arrives as a reviewable pull request, never a silent overwrite. Drift is not an embarrassment to hide — it is a bug to file and fix. Get those habits right and freshness stops being a heroic quarterly cleanup and becomes a background hum.

Front-matter: the metadata that does the work

Every entity in the catalog carries a YAML header. It is not decoration — each field is load-bearing, read by a skill or a human at a specific moment.

---
owner: trade-finance-pod
status: authoritative          # draft | reviewed | authoritative
last_verified: 2026-06-18
source_of_truth: db/migrations/  # what this doc is supposed to mirror
related_stories: [PROMO-488, PROMO-512]
---
FieldWhat it isHow the Librarian uses it
ownerThe team or person accountable for the entity.Who the docs PR gets assigned to and who is pinged when the entity goes stale.
statusHow much to trust it: draft, reviewed, or authoritative.Gates how the Analyst cites it — an authoritative rule is treated as fact; a draft is flagged as provisional.
last_verifiedThe date a human or the reconcile loop last confirmed the entity against reality.The clock the freshness audit watches. Past threshold → stale.
source_of_truthA pointer to the code or system the doc mirrors — a migrations directory, a job's source file, an integration contract.The reference the Librarian diffs against to detect drift. If the source changed after last_verified, the doc is suspect.
related_storiesThe Jira keys that created or changed this entity.The trail back to why the entity says what it says — provenance for every claim.

Two of these do most of the heavy lifting. source_of_truth turns "is this doc current?" from a judgement call into a comparison: point at db/migrations/, and a doc whose last_verified predates the newest migration is, by definition, suspect. related_stories turns every sentence into something traceable — you can always walk from a claim back to the Jira key that justified it.

The Karpathy-wiki philosophy

The catalog is a living wiki, not a spec. Specs get written once, approved, and then quietly diverge from the system they describe. A living wiki is edited continuously, in small increments, each one tracking a real change in reality. The model is the "Karpathy wiki": never a frozen rewrite, always append-and-refine — a thousand small diffs, each one reviewable, rather than one heroic document that is correct on the day it is signed and wrong forever after.

This is why the Librarian works in pull requests. A small diff — one column added to schema.md, one sentence sharpened in a domain rule — is easy to review, easy to trust, and easy to revert. A 400-line "documentation refresh" is none of those things, and nobody reads it. The Librarian deliberately keeps its edits small enough that a human reviewer can hold the whole change in their head.

Small diffs are a trust mechanism. The point of incremental edits is not tidiness — it is reviewability. A reviewer who can see exactly what changed, and why, against which merged PR, will approve in seconds and actually believe the result. That belief is the product. The Librarian optimizes every edit for it.

The reconcile loop

The Librarian's main job runs whenever the system changes: a story ships, a PR merges, a migration lands. That trigger fires the reconcile-docs skill, which finds the entities the change touched, updates them, refreshes their metadata, and opens a docs PR for a human to approve.

flowchart TB TRIG["Trigger<br/>PR merged · migration lands · story ships"] TRIG --> FIND["Find affected entities<br/>via impact-scan over the catalog graph"] FIND --> EDIT["Edit each entity<br/>small diff tracking the real change"] EDIT --> META["Bump last_verified<br/>link the Jira key in related_stories"] META --> PR["Open docs PR<br/>assigned to the entity owner"] PR --> REV{"Human review"} REV -->|approve| MERGE["Merge · catalog matches reality"] REV -->|changes requested| EDIT

The reconcile loop. A real change in the system fans out to the entities it touched; each is edited as a small reviewable diff, dated, and linked back to its Jira key before a human approves the merge.

Make it concrete. A migration lands on PromoDesk that adds a settled_amount column to the claims table — part of story PROMO-512, which lets the claims-matcher record how much of a retailer deduction was actually settled against a promotion. The Librarian does not touch one file; it traces the ripple:

$ reconcile-docs --trigger pr#1042 (PROMO-512)

affected entities (impact-scan over catalog):
  docs/data/schema.md              ← migration touched claims table
  docs/data/conceptual-model.md    ← "claim" gains a settled-amount concept
  docs/rules/claim-matching.md     ← settlement now recorded, not just matched

proposed edits:
  schema.md            + claims.settled_amount  numeric(14,2) NULL
  conceptual-model.md  + a claim now carries the amount settled vs. accrued
  claim-matching.md    + tolerance note: settled_amount may be partial

metadata on all three:
  last_verified: 2026-06-18 → 2026-06-26
  related_stories: + PROMO-512

→ opened docs PR #1043, assigned trade-finance-pod, 3 files, +11 −2

Notice the spread. A schema change is never just a schema change: it shifts the human-language conceptual model (a claim now has a settled amount) and it touches a domain rule (settlement can be partial, which the matcher's tolerance logic has to acknowledge). One real change, three entities, one dated and Jira-linked PR. That is reconcile working as intended.

Drift detection and the freshness audit

Reconcile handles changes the Librarian sees go by. Drift is the changes it missed — a migration that landed without triggering reconcile, a batch job whose code was edited but whose doc was not. The freshness-audit skill hunts for it by comparing each entity's source_of_truth against its last_verified date.

The rule is mechanical: if the source of truth has commits newer than the doc's last verification, the doc is stale until proven otherwise. Add a plain age threshold — anything not verified in 90 days — and the audit produces a report.

freshness-audit — PromoDesk catalog — 2026-06-26
────────────────────────────────────────────────────────────────
entity                       last_verified   source_of_truth        verdict
docs/data/schema.md          2026-06-26      db/migrations/          ok
docs/rules/claim-matching.md 2026-06-26      jobs/claims_matcher.py  ok
docs/batch/accrual-posting   2026-02-11      jobs/accrual_post.py    STALE  (source +3 commits)
docs/integrations/erp-gl.md  2026-03-30      contracts/erp_gl.yaml   stale  (88d, threshold 90d soon)
docs/glossary.md             2026-05-02      —                       ok
docs/security/authz.md       2026-01-09      —                       STALE  (no source, 168d old)
────────────────────────────────────────────────────────────────
2 stale, 1 ageing. opened tracking issues, pinged owners.

The accrual-posting entry is the dangerous kind: its source job has three commits newer than the last verification, so the documented behavior of the nightly accrual-posting run almost certainly no longer matches the code. The audit does not guess what changed — it raises the entity, names the owner, and lets reconcile (or a human) do the actual update. A finding, not a fix.

Drift is detected, not assumed. The audit never silently rewrites a stale entity from the source code — that would be the Librarian inventing facts. It flags the gap and routes it. Entities with no source_of_truth (the glossary, the authz matrix) can only be aged out by the calendar and re-verified by a human; the audit is honest about which kind of staleness it found.

Provenance: every claim traces to a change

This is the design rule from chapter 2 made operational — the catalog is the contract. The Librarian never invents facts about PromoDesk. Every edit it makes is traceable to a real change: a merged PR, a landed migration, a human-confirmed correction. That is what related_stories and the source-of-truth pointer are for — they are the citation trail.

And when the Librarian doesn't know, it says so. "The catalog doesn't cover how partial settlements interact with the ERP export yet" is a legitimate, useful answer — it is a finding that points at a gap, not a failure to paper over. An honest "I don't know, and here's where the hole is" keeps the catalog trustworthy in exactly the moments a confident guess would poison it.

Curation: keeping the catalog coherent

Beyond tracking changes, the Librarian keeps the catalog readable as it grows. This is the slow, gardening work:

When the human and the code disagree

The hardest case is contradiction: a stakeholder insists a promotion over $50k needs director approval, but the authz code in the source of truth enforces $75k. The wrong move is for the Librarian to quietly pick one and write it down — that buries the conflict and corrupts the catalog with a guess.

Instead the Librarian flags the contradiction. It records both claims, marks the entity's status down from authoritative, links the human's assertion and the code reference side by side, and routes it to the owner to resolve. A documented disagreement is honest and actionable; a silently resolved one is a landmine. The Librarian's job is to surface reality, including the parts where reality is in dispute.

Never resolve a conflict by fiat. Human-says-X-code-says-Y is the single most valuable signal the catalog can produce — it usually means a real bug, a stale rule, or a missing decision. Picking a winner destroys that signal. The Librarian's only correct action is to make the contradiction loud.

What this buys, and what's next

Front-matter that does real work, a reconcile loop tied to every merge, a freshness audit that diffs docs against their source of truth, provenance on every claim, and the discipline to flag conflicts instead of hiding them — together these turn the catalog from a wiki that rots into a wiki you can stake decisions on. The Librarian keeps the promise so that the Analyst, in the next chapter, can grill new requests against documentation it is safe to trust.