Hybrid Architecture vs Pure Attention: An Analysis of the Underlying Mechanisms of Large Model Reasoning Capabilities

This article compares the reasoning performance of hybrid architectures (attention + recurrence) and pure Transformer models, revealing that reasoning capabilities are based on two fundamental primitives—recall and state tracking. It finds that explicit reasoning training can expand the model's effective working range, but its benefits depend on the architecture's support for persistent state propagation; hybrid architectures are more robust in long-range state tracking tasks.

Large model reasoning capabilities have expanded from text completion to complex deduction, but the underlying mechanisms lack systematic research. Mainstream views treat reasoning as a single capability emerging from scale and data, ignoring basic cognitive primitives. This study, from the perspective of cognitive science, deconstructs reasoning into basic primitives and explores the differences in how different architectures support primitive capabilities.

Recall Primitive: Retrieve relevant information from long-range context (e.g., key information from earlier text, intermediate conclusions), similar to human working memory retrieval; State Tracking Primitive: Maintain the update and evolution of dynamic states during reasoning (e.g., variable changes); The two interweave to support complex reasoning (e.g., multi-step math problems require recalling initial conditions + tracking variable changes).

Architecture Comparison: Use two variants of the Olmo3 series (pure Transformer, hybrid architecture), with consistent parameter count, training data, and steps; each architecture has an instruction-tuned version and a reasoning-enhanced version (2×2 design); Experimental Tasks: Design state recall tasks (requiring state tracking + information recall), with difficulty graded by sequence length, number of variables, and transition complexity; Observation Metrics: Accuracy curves as difficulty changes, error pattern analysis, relative performance of architectures.

1. Reasoning-enhanced training yields the most significant improvement: Expands the model's effective working range, explaining the advantages of reasoning models like DeepSeek-R1;
2. Hybrid architectures are more robust for long-range dependencies: In long-sequence state tracking tasks, pure Transformer performance drops sharply, while hybrid architectures remain stable;
3. Architecture and training interaction: The effect of explicit reasoning training depends on the architecture's support for persistent state propagation, and the two complement each other.

Reasoning capabilities are supported by three levels of mechanisms:

1. Algorithm Layer: Explicit reasoning training (e.g., Chain of Thought) provides high-level strategies;
2. Architecture Layer: Network structure determines the efficiency of primitive implementation;
3. Representation Layer: Internal representations affect information storage, retrieval, and update; The three levels are interdependent, and architectural limitations can become bottlenecks for the algorithm layer.

Practical Guidance:

• For tasks requiring state tracking (multi-turn dialogue, planning), prioritize hybrid architectures;
• Reasoning training is not a panacea; architecture improvements need to be made simultaneously;
• Evaluation should cover difficulty gradients to avoid simple metrics masking limitations; Limitations and Future Work: Experiments are limited to a small range of model sizes and tasks; future work needs to expand validation and explore optimal parameters for hybrid architectures.