SIMMER: Uncovering Hidden Failures in LLM Planning—Blind Spots in Robotic Task Planning

The SIMMER benchmark focuses on hidden failures in LLM robotic task planning. Through systematic evaluation using a kitchen scenario world model, it finds that up to 56% of plans from state-of-the-art LLMs contain hidden failures, and counterfactual forward simulation can reduce the failure rate by 72%. This study fills the gap in LLM planning evaluation and provides important references for the safe deployment of AI agents.

Hidden failure is a covert and dangerous type of failure in LLM planning. Unlike immediate failure (which causes an error immediately during execution), it does not interrupt execution but undermines goal achievement, and may even lead to irreversible damage. For example: when a robot makes breakfast, boiling eggs first then placing the kettle causes eggshells to crack— the task seems completed but the result is inedible.

SIMMER constructs a semantically realistic symbolic world model for kitchen scenarios, including 77 actions, 262 unique objects, and approximately 46,800 real interactions (derived from cooking scripts). Equipped with a state machine executor, it can detect three types of failures: immediate premise violation, hidden danger, and irreversible failure, enabling precise analysis of failure patterns.

Experiments on six LLMs show: the highest error-free plan rate is only 17%, over half (56%) of plans contain hidden failures, and most hidden failures lead to irreversible consequences. This indicates that current LLMs are far from meeting the reliable deployment standard for planning in home environments.

The study proposes a counterfactual forward simulation solution, allowing the model to simulate action consequences before execution to identify risks. The experimental results are significant: hidden failures are reduced by 72% (from 56% to 16%), and irreversible cases are reduced by 75%, pointing the way for building robust LLM planners.

Insights from the SIMMER study for AI agent development: 1. Success rate is not the only metric; attention must be paid to hidden failure detection. 2. LLMs need to understand causal relationships in the physical world—world models and counterfactual reasoning are key. 3. Safe deployment requires multi-layered protection such as simulation testing, constraint checking, and human supervision.

SIMMER fills a key gap in LLM planning evaluation, systematically reveals the problem of hidden failures, and demonstrates the feasibility of improving explicit state reasoning. It provides an evaluation tool and reference framework for home AI agent developers. In the future, the reliability and safety of LLMs will be the key to their real-world application.