My website said I had delivered over a hundred and fifty projects. It said I had a satisfaction rate in the high nineties. It said I had over a decade of experience and had worked with hundreds of businesses across various industries.
None of that had happened. I am one engineer with one paying client.
I did not write those sentences and I did not catch them by reading. A model generated the copy, I read it, it sounded like a website, and it went live. I found them weeks later while auditing something unrelated — four separate fabrications, sitting on the page whose entire job is making a stranger trust me.
Why reading does not catch this
Because the invented numbers are the ones that read best.
A model writing marketing copy is doing exactly what it was asked. Marketing copy contains confident figures — that is the genre. Asked to produce copy for a software consultancy, a plausible project count and a plausible satisfaction percentage are not errors in its output. They are the shape of the thing.
And when you read it back, a number does not announce itself as unsourced. There is no visual difference between a figure you verified last month and a figure that was invented forty seconds ago. Both are just digits in a sentence you are skimming for tone.
That is the whole problem. The failure is invisible at exactly the moment you are supposed to catch it.
Why a better prompt does not fix it
My first instinct was to add "do not invent statistics" to the system prompt.
That is a wish. It works most of the time, which is the worst possible property — frequent enough to build confidence, unreliable enough to eventually ship. And you have no way to know which draft is the one where it did not hold, because the failure looks identical to the success.
Then I tried the obvious next thing: ask a model to check the copy. Is anything here fabricated?
This does not work either, and the reason is worth stating plainly. It is the same generator, asked a different question. It has no privileged access to whether 150 projects is true — it never did. It will read a plausible number, find it plausible, and approve it. You have added a second unmeasured system on top of the first one and called it verification.
What actually works: an allowlist
The fix is boring and mechanical.
A register of every figure the copy is allowed to contain, each with a source precise enough to re-verify:
[[fact]]
id = "haslett-invoiced"
scope = "public"
value = "$216K"
claim = "invoiced through software I wrote and still operate"
source = "Haslett Handyman master CRM, cumulative invoiced total"
stale_after = "2026-12-31"
[[fact]]
id = "audit-sites"
scope = "public"
value = "456"
claim = "Michigan wellness websites audited"
source = "places-leads sweep + site-audit, Michigan wellness vertical, 2026-08"And a checker with no model call in it. It extracts every number from the prose, normalises it, and fails on anything not in the register. Money, percentages, counts of three digits or more, multipliers, and written-out quantities like a third.
NUMBER = re.compile(
r"""(?<![\w/#-])(
\$\s?\d[\d,]*(?:\.\d+)?\s*[KkMm]? # $155K, $2,500, $1.5M
| \d[\d,]*(?:\.\d+)?\s?% # 37.5%, 0.4%
| \d[\d,]{2,} # 3,235 / 2407 — 3+ digits
| \b\d+\s?(?:x|×)\b # 90x
)(?![\w-])""",
re.VERBOSE,
)Plus a second pattern for the quantities that are claims without being digits — a third, dozens, hundreds, two thirds. Those slip past a numeric check and do exactly the same work in a sentence.
What it deliberately does not flag
This part took longer to get right than the matching did.
Numbers inside code blocks are stripped before the check runs. A port number, an HTTP status, a line of configuration — none of those are claims about a business, and a checker that argued about them would be wrong in a way that teaches you to stop reading it.
For the same reason there is a small allowlist of numerals that are never claims: the digits one through ten, common durations, the current year, and HTTP status codes. That last one is not tidiness. Status codes are three digits, so the fabrication rule catches every one of them, and without the exception an engineering post about a 404 fails on its own subject matter.
A linter people learn to ignore is worse than no linter. Every exception in that list exists because a false positive is not a small cost — it is the thing that turns the check into noise, and noise gets skipped exactly when it finally has something true to say.
The allowlist is per post, not global
The register holds every figure the business can prove. A given draft may cite only the subset its topic declares.
That rejects figures that are true. A post about whether clinic phone numbers can be tapped on a mobile has no business quoting revenue, and it is not stopped because the revenue number is wrong — it is stopped because a number wandering in from another story is how a draft quietly becomes an advert.
And a draft whose topic is not registered at all does not pass. It fails, with the reason written out: no topic means no allowlist, and no allowlist means nothing can be verified. An absent check has to read as a failure, because otherwise the easiest way to satisfy the system is to not be in it.
The second job: numbers that are real and still must not ship
Some facts in the register are marked internal. Pipeline counts, client counts — true, sourced, and nobody else's business.
The checker fails a draft that contains one of those figures, and names it. That is the opposite failure from the one the register was built for, and it needs the same machinery: the first check asks whether a number is real, the second asks whether it is mine to publish.
And one last thing it catches
The copy that started all of this was written in the first person plural. It said what the team had delivered, and how many clients the company had served.
There is no team. There is one engineer.
So the linter also flags we, our and us — as a warning inside quotation marks, since quoting someone else is legitimate, and as a failure everywhere else. It is the same class of problem as the invented project count: not a lie anybody chose to tell, just the default voice of the genre, arriving unexamined in copy nobody had a reason to doubt.
What it costs
It is a few hundred lines of Python with no model call in it, and it runs in under a second on a draft.
The register itself is the real work, and it is not a one-off — a figure goes stale, and a stale entry is worse than a missing one, because the checker keeps approving a number nobody has re-verified. Each fact carries its source written precisely enough to run again, which is the only part of this that makes the maintenance survivable.
What I get for it: I cannot publish a number I have not sourced, even when the draft reads perfectly and I am not paying attention. That is the point. The failure this prevents is invisible at the moment of review, so the check cannot depend on me being careful at the moment of review.
The same instinct, pointed at something smaller: the free website check reads your live pages and tells you what is actually on them. No signup, about a minute. People are surprised more often by what it finds than by what it misses.
If you publish numbers your business depends on — a case study, a pitch deck, a pricing page — and nothing checks them before they ship, that is a register waiting to be built. Tell me what you publish, and if your review process already catches it, I will tell you that instead.
