Reasoning Primitives of Hybrid Architecture LLMs: A Decoupled Analysis of Retrieval and State Tracking

Recent research decomposes the reasoning capabilities of LLMs into two fundamental primitives: retrieval (retrieving information from trained knowledge) and state tracking (maintaining and updating intermediate states). The study finds that hybrid architectures (combining attention-based retrieval and cyclic state updates) significantly outperform pure attention models in state tracking tasks without sacrificing retrieval ability. This discovery provides new ideas for selecting appropriate architectures for different application scenarios, promoting the understanding of LLM reasoning capabilities from a black-box to a white-box approach.

In the past, the reasoning capabilities of LLMs were often viewed as a single, indivisible whole, discussed as a black box (either present or absent). This perspective obscures the complex mechanisms behind reasoning. Recent research suggests that observed reasoning gains may stem from more fundamental cognitive operations rather than a mysterious "reasoning module", thus requiring decomposition into analyzable primitives for study.

The study identifies two key reasoning primitives:

• Retrieval: Retrieve relevant information from trained knowledge (similar to long-term memory extraction)
• State Tracking: Maintain and update intermediate states during sequence processing (similar to working memory)

Two architectures are compared:

• Pure attention Transformer model
• Hybrid architecture (attention + cyclic state updates)

The experiment uses matched Olmo3 Transformer and hybrid variants, comparing them under instruction fine-tuning and reasoning enhancement configurations to ensure that differences stem from architecture rather than other factors.

1. Architecture Performance: Hybrid architectures significantly outperform pure attention models in state tracking tasks without sacrificing retrieval ability.
2. Task Adaptation:
   • Complex state maintenance tasks (multi-step logical reasoning, long-range dependencies): Hybrid architectures are better
   • Knowledge retrieval tasks: Both perform similarly
3. Benchmark Contribution: Different reasoning benchmarks rely on retrieval and state tracking to varying degrees; a single benchmark score cannot fully evaluate reasoning capabilities.

AI system designers can select architectures based on the task's requirements for primitives:

• Question answering/knowledge retrieval: Pure attention architectures are sufficient
• Code generation/mathematical reasoning/multi-turn dialogue: Hybrid architectures are more appropriate
• General assistant systems: Need to dynamically select or combine different architectures based on specific scenarios.

Future Directions:

• Modular, task-oriented model design (explicit state management, configurable attention, dynamic architecture selection, etc.)
• Specialized training methods for specific primitives

Limitations:

• Conclusions are based on the Olmo3 model family and specific task sets; generalizability needs further verification
• The decomposition of retrieval and state tracking may be overly simplified; real reasoning may involve more cognitive primitives

Future research can explore other primitives, primitive interaction mechanisms, and multi-capability integration methods.