When Large Models Stop Following Steps: A Diagnostic Study on the Procedural Execution of Language Models

This study, through controlled diagnostic benchmark tests, found that large language models have significant capability defects in procedural execution tasks: when the number of program steps increases from 5 to 95, the average first-answer accuracy rate plummets from 61% to 20%. Failure modes include missing answers, premature termination, etc., revealing the issue of execution faithfulness behind superficial reasoning abilities.

Large language models perform well in benchmark tests such as mathematical problem-solving and logical reasoning, but an overlooked question is: does the correct answer come from faithful execution of instructions? This study questions this point, designs a procedural execution diagnostic benchmark, and reveals the substantive execution defects behind superficial reasoning abilities.

The study chose arithmetic programs as the test carrier (verifiable, simple, controllable), and controlled complexity from two dimensions: 1. Program length (5-95 steps, testing long-range dependencies); 2. Lookback dependency (referencing intermediate variables to simulate state transfer in real algorithms).

Testing 14 models and 55 configurations, the results show: 5-step program accuracy is 61%, 95-step drops to 20%. The main failure modes are five types: missing answers, premature answers, self-correction after errors, traces of insufficient execution, and hallucinated extra steps.

This defect poses a reliability crisis for key applications such as financial computing and medical decision-making. Traditional end-to-end evaluation may mask problems; it is recommended to adopt fine-grained evaluation methods such as process supervision, adversarial testing, and length extension testing.

1. Autoregressive generation is prone to error propagation; 2. The Transformer attention mechanism dilutes early information as the sequence grows; 3. Most programs in training data are natural language descriptions, leading to approximate execution rather than precise execution.

Architecture level: explicit state maintenance, structured generation, validator integration; Training strategy: program synthesis data, reinforcement learning (process rewards), curriculum learning (from short to long programs).

Current limitations: only targeting arithmetic programs, limited model scope, not deeply exploring the impact of prompt engineering. Future directions: expanding to multimodality, studying the relationship between scale and faithfulness, developing automated evaluation tools. Conclusion: True reasoning requires faithful adherence to the process; a predictable system is more valuable than one that is occasionally correct but unexplainable.