SkepticalSam
FactSelfCheck: Can Knowledge Graphs Catch Hallucinated Facts?
FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs
Published: 12 May 2026 · Updated: 13 July 2026
Read the original sourceWhat the paper says
FactSelfCheck converts long-form outputs into fact triples or knowledge graphs, samples multiple responses, and checks consistency across them. It aims to detect hallucinated facts without external databases or model internals.
The Critique
Fact-level detection is the right target, but the method depends on extraction quality. If the system extracts the wrong triples, it can score the wrong facts. It also assumes hallucinations vary across samples. Some false claims are stable because they are memorised, common, or reinforced by the prompt. A consistent hallucination can pass a consistency check.
Why It Matters
Fine-grained hallucination detection would be much more useful than sentence-level warning labels. Users need to know which claim is suspect, not that a paragraph feels risky.
What They Missed
Robustness against stable falsehoods, long-document scaling, human review of extracted triples, and comparisons with retrieval-based verification.
The Big Question
Can a model catch hallucinations by comparing itself to itself, or do consistent falsehoods slip through the net?