CoDE-Stop: Let Large Models Learn to "Stop in Time", Boost Reasoning Efficiency by 50%

This article introduces the CoDE-Stop method, which aims to solve the "overthinking" problem in large model reasoning. By monitoring confidence dynamics during reasoning, the method allows the model to stop thinking early when confidence is high and stable, saving 25-50% of computational costs while keeping accuracy essentially unchanged.

Large model reasoning relies on long chains of thought to solve complex problems, but there are two major issues: 1. Soaring computational costs (unnecessary token generation); 2. Performance degradation (overthinking leads to deviation from the correct answer). Studies have found that in correct reasoning trajectories, answers often appear early with stable confidence, while in incorrect trajectories, confidence fluctuates erratically.

Core idea of CoDE-Stop: Stop reasoning when the model's confidence in the answer is sufficiently high and consistently stable. Working mechanism: 1. Monitor intermediate answers; 2. Calculate confidence; 3. Analyze confidence dynamics; 4. Trigger stopping (high confidence + stability conditions). Advantage: No additional training required, plug-and-play.

Experiments show: 1. Compared with full-length reasoning, token usage is reduced by 25-50% while accuracy remains essentially unchanged; 2. Outperforms existing methods such as fixed steps, single confidence threshold, and perplexity; 3. Effective across models of different architectures, with strong universality.

• Correct trajectories: Confidence rises rapidly and stabilizes; Incorrect trajectories: Fluctuates and remains low. - Stop point distribution: Early stopping for simple problems (20-30% tokens), mid-to-late stopping for complex problems (50-70% tokens), and near the upper limit for very few difficult problems. - Cost of overthinking: 15% of cases change answers, 60% switch from correct to incorrect.

Applicable to: 1. Online reasoning services (reduce costs, improve response speed); 2. Resource-constrained environments (edge/mobile devices); 3. Real-time applications (dialogue systems, real-time recommendations); 4. Batch processing (data analysis, document processing).

Limitations: Relies on the accuracy of confidence estimation. Future directions: 1. More precise confidence estimation; 2. Task-specific hyperparameter tuning; 3. Internalizing stopping capability into models; 4. Extending to tasks like long text creation and code generation.

CoDE-Stop represents progress in optimizing the reasoning efficiency of large models, emphasizing the smart use of computational resources rather than simply increasing scale. Let AI learn to "stop in time" and move toward more intelligent and practical AI systems.