Reasoning Collapse: The Capability Boundaries of Large Models in Complex Reasoning Tasks

This article reveals the phenomenon of "reasoning collapse" in large reasoning models (LRMs) when facing problems of increasing complexity through systematic testing of nine classic reasoning tasks—accuracy drops sharply after exceeding a specific threshold, challenging the overly optimistic expectations of their reasoning capabilities.

Large language models (especially LRMs with explicit reasoning capabilities) perform well in benchmark tests such as mathematics and logic, spawning the view that "AI reasoning capabilities are comparable to humans". However, existing evaluations rely on aggregate accuracy of fixed datasets, which masks the evolution of model performance when task complexity increases.

The study selected nine classic discrete state space problems including Boolean Satisfiability (SAT), cryptarithmetic, and Tower of Hanoi. Complexity was precisely controlled by adjusting parameters (e.g., number of disks in Tower of Hanoi, number of scrambling steps for Rubik's Cube), and a deterministic verifier was used to ensure strictly correct results.

All tested models showed a consistent pattern: high accuracy on low-complexity tasks, but a steep drop in accuracy after exceeding task-specific thresholds (some dropped from nearly 100% to 0%). Collapse also manifests in inconsistent reasoning traces, constraint violations, loss of state tracking, and confident incorrect outputs.

Reasoning collapse is a cross-model universal phenomenon (thresholds vary by model/task); increasing reasoning length does not necessarily improve correctness (superficial plausibility ≠ substantive correctness); models rely on pattern matching rather than transferable reasoning strategies (generalization failure); existing static benchmarks overestimate capabilities, requiring complexity-aware evaluations and strict verification.

1. Maintain realistic expectations for model reasoning capabilities; 2. Strict result verification is required for critical applications; 3. Decompose complex problems into low-complexity subproblems.

Possible causes: Limitations of Transformer attention mechanisms, training data biased towards simple instances, confusion between reasoning and memory. Future directions: Develop architectures with external memory, optimize training strategies for samples with increasing complexity, and integrate hybrid methods combining neural networks and symbolic systems.