SkepticalSam
Poverty of Stimulus — 'Above-Chance' ≠ 'Acquired'
A Unified Assessment of the Poverty of the Stimulus Argument for Neural Language Models
Published: 12 May 2026 · Updated: 13 July 2026
Read the original sourceWhat the paper says
Models show above-chance performance on Poverty of Stimulus tests.
The Critique
The nuanced finding: models do something, but not enough! This refutes strong claims on both sides—neither 'LLMs have fully acquired syntax' (overclaim) nor 'LLMs completely fail at syntax' (underclaim) is correct. It reveals evaluation methodology matters.
Why It Matters
The gap between above-chance and mastery matters for claims about neural network language acquisition. 'Better than random' is not 'as good as humans.'
What They Missed
Using perplexity-based preference vs actual grammatical judgments gives different results. This null result for strong acquisition teaches us about evaluation design.
The Big Question
What would it take for neural networks to truly acquire human-like linguistic competence?