Diagnosis of Quantization Failure in 2-bit Reasoning Models: Cycle Rescue Mechanism Boosts Qwen3-8B Accuracy from 17.2% to 74.2%

Paper Core: 2-bit quantization leads to generative pathologies such as loops and delayed commitments in reasoning models. Two lightweight control methods—FP16 planning and cycle rescue—are proposed, which boost Qwen3-8B's accuracy from 17.2% to 74.2% while maintaining end-to-end speed.

Original Paper Information:

• Author Team: Brain Lab Research
• Source: arXiv
• Title: Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery
• Link: http://arxiv.org/abs/2606.02011v1
• Code Repository: https://github.com/brain-lab-research/quantized-reasoning
• Publication Date: 2026-06-01

Large reasoning models (LRMs) like Qwen3 and DeepSeek-R1 improve accuracy by generating reasoning traces, but incur huge inference costs (more tokens, high computational overhead, increased latency).

Low-bit quantization (2/3/4-bit) is a popular direction to reduce costs, but aggressive 2-bit quantization not only reduces accuracy but may also fail to bring end-to-end acceleration—because the stability of the generation process is disrupted.

2-bit quantization triggers generative pathologies, including:

1. Repetitive Loops: Getting stuck in identical/similar content loops and failing to progress;
2. Budget Exhaustion: Reaching the maximum length limit, leading to incomplete reasoning;
3. Delayed Commitment: Over-analyzing and wavering, adding unnecessary steps;
4. Unclosed Reasoning Segments: Structural chaos (e.g., mismatched parentheses).

Key Insight: The accuracy drop is closely related to these process-level failures, not the model being "dumber". Fixing generation stability can restore accuracy.

To address generative pathologies, two lightweight control methods are proposed: 1. FP16 Planning:

• Core: Use an FP16 model to generate a short reasoning outline, which is used as conditional input for the 2-bit model to fill in details;
• Advantages: Lightweight, effectively anchors the generation direction, and allows flexible adjustment of outline detail level.

2. Cycle Rescue:

• Core: Detect repetitive patterns, with intervention strategies including early submission (directly output when a reasonable answer is available) or FP16 fallback (switch to FP16 to continue generation);
• Detection Mechanisms: n-gram repetition detection, semantic similarity comparison, generation pattern analysis.

MATH-500 Benchmark Results:

Configuration	Qwen3-8B	Qwen3-32B
2-bit Baseline	17.2%	65.0%
+ Cycle Rescue	74.2%	-
+ Planning + Cycle Rescue	-	87.2%

Key Conclusions:

• Qwen3-8B's accuracy increases by 57 percentage points, approaching FP16 performance;
• End-to-end speed maintained: The overhead of lightweight control is minimal, and the reduced number of tokens offsets additional computation, making overall speed better than the uncontrolled baseline;
• Consistent performance across tasks (math/common sense), with strong universality.

Conclusions: This study reveals that aggressive low-bit quantization disrupts the stability of the generation process in reasoning models (rather than just reducing quality). Through two lightweight methods—FP16 planning and cycle rescue—accuracy is significantly restored while maintaining speed.

Future Directions:

1. Adaptive Quantization: Dynamically adjust precision based on content;
2. Learning-based Control: Optimize control strategies using learning methods;
3. Hardware Collaboration: Combine with specific hardware characteristics;
4. Multi-model Collaboration: Collaborative reasoning among multiple quantized models.