SkepticalSam
Aletheia's 'Autonomous' Math — The Hype vs. The Reality
Towards Autonomous Mathematics Research
Published: 12 May 2026 · Updated: 13 July 2026
Read the original sourceWhat the paper says
AI has achieved autonomous mathematical research capabilities, solving previously open problems and generating publication-worthy papers without human intervention in the proof process.
The Critique
The autonomy claims are carefully worded but misleading. They solved four 'open' Erdős problems—but admit most were 'quite elementary,' with one (Erdős-397) nearly identical to a 2012 Chinese IMO team selection test problem. The 'autonomous' eigenweights paper required humans to define the research question, set up the computational framework, and presumably select the result from many failed attempts. The authors acknowledge this is only 'Level 2: Publishable Research' on their own taxonomy—not 'Major Advance' or 'Landmark Breakthrough.' Yet the framing and media coverage will inevitably suggest more than this. Most concerning: they report solving 4 of 700 Erdős problems—a 0.6% success rate on problems that have resisted human mathematicians for decades. Is this because the problems are hard, or because the model lacks the research judgment to know which problems are tractable? The 'scaling laws' they show are on competition problems with known solutions—very different from open research questions.
Why It Matters
If the field overestimates AI's autonomous research capabilities, we might reduce funding for human mathematicians or deploy AI research tools before they're reliable. The 'autonomy levels' framework is useful, but only if the field uses it honestly.
What They Missed
They don't report how many computational resources were expended per successful result, how many attempts were made before success, or how much human intervention was required in problem selection. Without these baselines, 'autonomous' is meaningless. They also don't analyze the failed attempts on the 696 unsolved Erdős problems—understanding why the model failed might be more valuable than celebrating the 4 successes.