Thread
The Fabrication Problem
Most AI numbers are fabricated. Source material fixes it. Self-checking fails. Trust signals are backwards.
Answered Source material collapses unsourced numbers to single digits. Prohibition outperforms monitoring 5x. Self-checking fails because the same process generates and evaluates. Trust signals (citations, confidence, specificity) are higher in fabricated output than sourced output.
Open Does the fix work beyond reformulation tasks? Reasoning shows improvement (75% vs 38%), but strategy and creative untested.
Most AI Numbers Are Fabricated
77 to 100 percent of AI-generated numbers are temporally unstable. Source material fixes it. Prompts don't.
How to Stop AI from Making Up Numbers
Source material drops AI fabrication from 85% to single digits. Three steps.
Why AI Can't Check Its Own Work
The agent reported clean. The output was wrong. Same process generating and evaluating.
The Most Trustworthy AI Output Is the Least Reliable
The signals you use to judge AI trustworthiness are the same signals fabrication produces.