ReasonLite: An Ultra-Lightweight 0.6B-Parameter Reasoning Model Challenges 10x Larger Counterparts

AMD's open-source ReasonLite-0.6B achieves remarkable mathematical reasoning capabilities with 0.6B parameters, scoring 75.2 on AIME24 and surpassing 10x larger models like Qwen3-8B, providing new ideas for expanding the reasoning capabilities of small models. The open-source project includes model weights, training scripts, training data, and data synthesis/filtering processes.

Large models have strong reasoning capabilities but high deployment costs and limited application scenarios; small models are irreplaceable in scenarios like edge devices. The traditional view holds that small models have a capability ceiling in complex reasoning tasks, but ReasonLite breaks this impression, proving that high-quality data distillation can enable small models to achieve excellent reasoning levels. The AMD AGI team open-sourced all project resources, providing a reproducible research foundation for the community.

ReasonLite adopts a two-stage progressive distillation strategy: In the first stage, short CoT data is used to distill Qwen3-0.6B into the Turbo version, increasing AIME24 accuracy from 11.0 to 57.1; in the second stage, long CoT data is introduced for deep training to obtain the full version, further improving the score to 75.2.

The training data starts from high-quality sources like Polaris (53K) and OpenMathReasoning. Using GPT-OSS as the teacher model, 9.1 million raw answers are generated, then pseudo-labels are created via a majority voting mechanism, and 6.1 million high-quality samples are filtered out (4.3 million medium difficulty, 1.8 million high difficulty). This strategy proves that data quality is more important than scale in reasoning tasks.

ReasonLite performs excellently in multiple benchmark tests: It scores 75.2 on AIME24, surpassing Qwen3-8B (74.6) and Deepseek-qwen-14B (65.0); achieves 62.9 on AIME25, showing good generalization ability; gets 95.2 on AMC23, demonstrating solid basic reasoning; and reaches 90.2% in pass@8 on AIME24, which is of great value for answer verification and reordering in practical applications.

ReasonLite provides a feasible solution for reasoning deployment in resource-constrained scenarios, proving that there is still room for improvement in expanding the reasoning capabilities of small models. It offers developers a complete training and evaluation toolchain (open-r1 training code, DeepMath evaluation framework, data processing process), supporting ROCm and CUDA platforms, lowering the deployment threshold.

ReasonLite achieves a 75.2 score on AIME24 with 0.6B parameters, proving the potential of data quality, training strategies, and distillation technology. The open-source project allows the community to further explore and optimize, and more derivative models may emerge in the future to promote the progress of small model reasoning capabilities.