DOT: Dynamic Outlier Truncation Technology Empowers Efficient Reasoning Model Training

DOT (Dynamic Outlier Truncation Technology) is the official code implementation of a paper accepted by ACL 2026. It proposes a dynamic outlier truncation method to address the "length bias" problem in reasoning model training. While maintaining model performance, it significantly improves training efficiency and stability, providing a new solution for reasoning model training optimization.

Reasoning model training faces the challenge of "length bias"—models tend to generate lengthy reasoning chains, leading to low training efficiency, high reasoning costs, and unstable convergence. Existing methods (length penalty, hard truncation, curriculum learning) use fixed strategies that cannot adapt to dynamic data distributions, resulting in limited effectiveness.

The core of DOT is the dynamic outlier truncation idea: 1. Outlier detection mechanism (real-time tracking of length distribution, dynamic threshold calculation, context awareness); 2. Selective truncation strategy (retaining effective long chains, truncating redundant short chains, gradient reweighting); 3. Online adaptation mechanism (real-time statistical update, sliding window estimation, smooth transition). In terms of technical implementation, it uses robust statistics (median + interquartile range), comprehensive truncation decision logic (relative length + task difficulty + training stage), and training stability guarantees (progressive introduction, temperature annealing, validation set monitoring).

In benchmark tests such as mathematical reasoning (GSM8K, MATH), code generation (HumanEval, MBPP), and logical reasoning, DOT shows significant performance: the average reasoning length is reduced by 30-50% while maintaining or improving accuracy; training time is shortened by 20-40%, GPU memory usage is reduced, and convergence is accelerated; model quality is comparable to or slightly better than the baseline. Ablation studies verify the necessity of dynamic thresholds, quantile detection, and online adaptation.

DOT can reduce training costs (lowering computational resource consumption), improve user experience (shortening reasoning latency), and contribute to environmental friendliness (reducing energy consumption and carbon emissions), with direct commercial value and sustainability significance.

Current limitations include hyperparameter sensitivity, task specificity, and limited theoretical understanding. Future directions can explore adaptive hyperparameters, multi-task general strategies, theoretical analysis, and combination with technologies such as reinforcement learning/distillation.

DOT provides an effective solution to the length bias problem of reasoning models. While maintaining performance, it improves training efficiency. It not only has practical value but also provides a new perspective for understanding the learning dynamics of reasoning models, and will play an important role with the popularization of reasoning models.