NeuReasoner: An Interpretable, Controllable, and Unified Large Model Reasoning Framework via Mixture of Neurons

NeuReasoner proposes a unified reasoning framework based on Mixture of Neurons (MoN). It detects and fixes reasoning failures by identifying key neurons and their fluctuation patterns through white-box analysis. The framework achieves a maximum 27% performance improvement on six benchmarks while reducing token consumption by 19.6% to 63.3%. It addresses the three major challenges of Large Reasoning Models (LRMs): intra-step errors, inter-step oscillation/stagnation, and instance-level overthinking, and it has both interpretability and controllability.

Large Reasoning Models (LRMs) like DeepSeek-R1 have made significant progress in complex reasoning tasks, but they have three failure modes: intra-step calculation/deduction errors, inter-step oscillation/stagnation, and instance-level overthinking. Most existing studies optimize for a single aspect and rely on black-box RL training, lacking a unified solution, which limits interpretability and controllability.

The research team identified key neuron groups associated with different failure modes—Mixture of Neurons (MoN)—through white-box analysis. Different reasoning failures correspond to unique activation fluctuation patterns of specific neuron sets (e.g., calculation errors are related to abnormal activation of numerical processing neurons). Based on this, the NeuReasoner framework was proposed to detect and fix reasoning failures in real time by monitoring key neuron activities.

NeuReasoner consists of two core components: 1. Lightweight MLP failure detector: Monitors MoN activation patterns in real time, quickly identifies potential failures, and is efficient and interpretable; 2. Special token-triggered self-correction mechanism: Inserts predefined special tokens when failures are detected, and enables the model to execute corresponding repair strategies (such as recalculation or changing reasoning paths) through Supervised Fine-Tuning (SFT), achieving real-time monitoring and dynamic adjustment.

Evaluated on 6 benchmarks (including mathematical reasoning, code generation, etc.) and 6 backbone models of different scales (8B-70B), NeuReasoner achieves a maximum 27.0% performance improvement compared to 9 competing baselines, while reducing token consumption by 19.6% to 63.3%. The results are consistent across model scales, showing good scalability and generality.

Theoretically, it is the first time to systematically link LRM failure modes with neuron activities, providing a new direction for the study of internal mechanisms of black-box models; Practically, it provides a reliable reasoning technical path. Its lightweight design makes it easy to deploy, its controllability supports scenario customization (such as medical and financial fields), and it optimizes resource consumption by avoiding overthinking, making it suitable for resource-constrained environments.

NeuReasoner constructs a unified and controllable reasoning framework through white-box analysis, achieving excellent performance and efficiency. In the future, we can extend the MoN concept to more models/tasks, explore neuron groups corresponding to different cognitive functions, deepen the special token control paradigm, and promote the development of more reliable, efficient, and controllable intelligent reasoning systems.