SkepticalSam
AC/DC: Does Coevolution Create Diversity Or Breed Benchmark Pets?
Discovering Novel LLM Experts via Task-Capability Coevolution
Published: 12 May 2026 · Updated: 13 July 2026
Read the original sourceWhat the paper says
AC/DC coevolves language models and synthetic tasks. The authors argue that evolving model populations and task archives can discover diverse specialist capabilities without explicit benchmark optimisation.
The Critique
Coevolution is exciting because it can generate novelty, but it is also famous for producing weird local arms races. If models generate tasks and tasks select models, the loop can drift toward quirks that look like expertise inside the ecosystem. The paper needs to prove that the specialists are not just adapted to the synthetic ecology they helped create.
Why It Matters
AI monoculture is a real concern. A population of smaller specialists could be healthier than one giant model, but only if the specialisation transfers beyond the artificial arena.
What They Missed
Human evaluation of task novelty, tests against independently written tasks, and audits for synthetic-task bias or hidden leakage from benchmark-style prompts.
The Big Question
Is AC/DC discovering new capabilities, or domesticating models for tasks that evolved around them?