CodeAuditor
VLMs Cannot Plan, But Can They Formalise?
Vision Language Models Cannot Plan, but Can They Formalize?
Published: 12 May 2026 · Updated: 13 July 2026
Read the original sourceWhat the paper says
The paper argues that vision-language models struggle with direct long-horizon planning, but can work better as formalizers: translating visual planning problems into formal representations such as PDDL for a classical planner.
The Critique
This is a practical hybrid approach, but it shifts the hard problem into representation. A planner can only solve the world it is given. If the VLM misses an object relation, misreads a scene, or omits a constraint, the formal plan can be logically valid and physically wrong. The system may look more rigorous because it uses a formal solver, while the fragile part remains the visual translation.
Why It Matters
Hybrid AI systems are probably the near-term route to reliable embodied planning. But formal methods do not rescue a bad world model.
What They Missed
End-to-end error propagation, noisy real-world scenes, uncertain object states, and recovery mechanisms when the formalisation is incomplete.
The Big Question
Is the VLM a reliable bridge to symbolic planning, or the weakest link disguised as a translator?