Guide · New to this · In progress
The five-tier verification ladder
The one idea underneath everything else here: self-review catches typos, internal consistency catches arithmetic, independent re-derivation catches logic, external authority catches framing, and a hostile expert catches what you were sure of.
Every result climbs the same five rungs, as far up as the stakes justify. Self-review catches typos and obvious slips. Internal consistency catches contradictions between one part of an answer and another. The workhorse rung is the third: a second implementation, written fresh without looking at the first, has to reproduce every number. Above that sits external ground truth: a published table, a live API, a reference database that already knows the answer. The top rung is a hostile expert, someone whose job for the next ten minutes is to find the hole.
Most people stop at rung one or two without noticing they stopped. An answer that reads clearly and sounds confident feels checked even when nothing has actually been re-derived. That feeling is the failure mode this whole guide exists to counter: the check that passed was never a check on the thing you actually needed to trust.
Across a full year of AI-assisted computational work, the pattern held up in an unexpected direction. Computed numbers essentially never turned out wrong once they were pushed up the ladder. What failed, at every tier, was narrative: a status line that said "running" long after the job had finished, a discovery sitting unflagged inside data that had already been verified correct, a cause-of-bug explanation that sounded mechanically plausible and was not the real cause. The numbers survived. The story around them did not.
Start today: pick the single AI-assisted output you trust the most right now, the one you would be most embarrassed to have wrong. Climb it one rung higher than you already did. If you only self-reviewed it, check internal consistency. If you already checked consistency, go find a second, independent way to produce the same number. The rung you skip is never random; it is always the one that would have caught the thing you already suspected.
First pass, not final
This entry is a first draft grounded in the program's real practice and incident record, awaiting Tyler's own rewrite before it's treated as finished, the same standard every piece of writing on this site and on Checkable holds to. Analysis and drafting assisted by Claude (Anthropic); every result independently verified by recomputation, external ground truth, or expert review.