There’s a category of software whose correctness isn’t defined by its own tests. Tax engines are right or wrong against the tax code. Dosing tools are right or wrong against clinical guidelines. Payroll, aviation maintenance, structural engineering, insurance rating — all of them are, at bottom, rulebooks compiled into code. The system I’ve been working on belongs to this category too: it turns user input into a detailed 3D model of physical assemblies, and every dimension in that model is right or wrong against an industry rulebook and a thousand-page technical handbook.
The problem with this category is that the source of truth is a stack of paper. The code was written years ago by people reading those documents, and every port, refactor, and quick fix since then has been drifting away from them — invisibly, because the automated tests only check that the code does what the code does. Nothing checks that the code does what the book says.
Large language models turn out to be genuinely good at exactly this gap. Not writing the code — reading the book. But only if you force a discipline on them that I want to describe here, because the naive version fails in a specific, dangerous way.
The naive version, and why it fails
The obvious approach: hand the AI your PDFs, ask “does my code match?”, get back a confident essay. The essay will be plausible, mostly right, occasionally wrong, and — this is the killer — uncheckable. When an AI says “your spacing logic deviates from standard practice,” you have three options: trust it blindly, spend an hour verifying it yourself, or ignore it. All three defeat the purpose.
The fix isn’t better prompting. It’s bookkeeping.
Claims as the unit of work
We rebuilt the process around one unit: the claim. The AI reads a chapter — as page images, because in technical documents the real information lives in the figures and tables — and emits atomic, cited facts:
CLAIM-book-p365-1: layout marks step back half a member thickness, so that centres land on exact spacing multiples and joints fall on member centres. (source: p. 365, procedure + figure)
Every claim then gets mapped against the codebase and stamped with one of three verdicts: matches (with file and line), deviates (with both values, the book’s and the code’s), or missing. The result is a ledger, not an essay. About a third of our claims came back matches — which is not wasted work; it’s the calibration that makes the deviations believable.
Three things happened that generic “AI code review” never produces:
Magic numbers explained themselves. Every legacy codebase has constants nobody can justify. Ours had one — an oddly precise length that appeared in two places with no comment. The book, on the page the original developers must have read, shows the arithmetic: it’s one term of a three-part assembly designed so that everything downstream lines up. The code had kept the number and lost the reason — and had been silently displaying the assembly wrong while calculating it right. One claim, one page number, a years-old mystery closed.
Sub-visible errors surfaced. A layout grid sat a fraction of a unit off its intended module. No screenshot would ever show it — but it’s the exact mistake the handbook warns about, which is why a procedure for it exists at all. The claim caught it because claims compare numbers, not appearances.
The reviewer became falsifiable too. Each claim carries a page number, so anyone with the book can check the AI’s reading. That cuts both ways, and it should.
Adversaries and arithmetic
Two more mechanisms earned their keep.
First: findings must survive attack. Every deviation the AI reported went to separate AI instances with one job — refute it. Redo the arithmetic, re-read the code, default to “refuted” when uncertain. Roughly a third of raw findings died there: misread context, exaggerated impact, or behavior that turned out to be intentional. What survived was worth a human’s time. The asymmetry matters: generating findings is cheap, so you can afford to be brutal about filtering them.
Second, for spatial or numeric output: verify computationally, not visually. After implementing fixes, we’d regenerate the full model and run thousands of geometric intersection checks across every component. Thirteen real defects came out of one such pass — parts occupying the same space, pieces floating just off their supports — all invisible in a rendered screenshot, all trivially provable with arithmetic. If your output has coordinates, your review should do math on them.
The last checkbox belongs to a human
The final artifact of the whole process wasn’t code. It was a review document written for a domain expert who doesn’t program: each entry says what the software does today, what the book says — page number included — and offers a checkbox: True / False, with room for “the rule is right, but practice has moved on” — a verdict only a practitioner can render.
That document reframes what the expert is doing. They’re not approving software. They’re checking claims about their own field, against a book they own, on pages they can open. That’s a review a non-technical person can actually perform — and it’s a far stronger guarantee than any amount of AI confidence.
The checklist version
- Feed the AI your sources — the documents your system was actually built from. The distance between “typical practice” on the internet and your spec is where the real bugs live.
- Demand claims with citations, never conclusions. One fact, one ID, one page number, one verdict against the code.
- Record the matches, not just the deviations. Confirmations calibrate trust and give reviewers something to spot-check.
- Attack every finding with independent refuters before a human sees it. Default skeptical.
- If the output is spatial or numeric, verify computationally — recomputed arithmetic and geometric checks, not screenshots.
- Protect one invariant absolutely (for us: the numbers customers ultimately pay for), and route anything touching it to a human decision.
- End at a falsifiable review: a domain expert, the source document, and a checkbox that says False.
The surprise of this whole exercise wasn’t that AI could do it. It’s which AI skill mattered. Not code generation — reading. Patient, cited, checkable reading of a thousand pages nobody on the team was ever going to re-read. The machine’s job was to keep receipts. The expert’s job was to judge them. That division of labor is, I suspect, what “AI verification” is going to look like everywhere.
