R2R: Efficient Reasoning Path Exploration via Collaborative Routing Between Small and Large Models

The NeurIPS 2025 paper R2R proposes a token routing mechanism for collaboration between small and large models to address the high inference cost of large models, significantly reducing computational costs (e.g., 40-60% cost reduction in math tasks) while maintaining reasoning quality.

Large models generate a large number of intermediate tokens in complex reasoning tasks (chain-of-thought, multi-path exploration), leading to exponential cost growth that restricts practical deployment. R2R aims to balance reasoning efficiency and quality.

Core Insight: Different tokens in reasoning have varying importance—key decision points require large models, while routine content can use small models. The architecture includes a routing strategy network (a lightweight classifier to predict token difficulty), a small model (for simple tokens), and a large model (for difficult tokens). Strategy learning uses a self-supervised approach: label difficult tokens using the large model's golden path to optimize the balance between accuracy and cost.

Mathematical Reasoning (GSM8K, MATH): Maintains similar accuracy with 40-60% cost reduction; Code Generation (HumanEval): Significant cost advantages, with slightly higher pass rates in some scenarios; Ablation experiments prove the learning strategy is effective, while random or fixed threshold strategies perform poorly.

Applicable Scenarios: Cost-sensitive online services, edge devices (local small model + cloud large model), multi-tenant systems (adjusted according to user preferences). Deployment Recommendations: Train the strategy using task data, and establish a monitoring mechanism to track quality and cost.

Limitations: Requires golden outputs from large models; multimodal reasoning remains to be explored. Future Directions: Weakly supervised/RL training strategies, multimodal expansion, multi-model systems, global path optimization. Conclusion: R2R provides an important direction for LLM inference optimization, and intelligent system design is key to practical application.