# Diagnosis of Quantization Failure in 2-bit Reasoning Models: Cycle Rescue Mechanism Boosts Qwen3-8B Accuracy from 17.2% to 74.2%

> The study reveals that 2-bit quantization causes generative pathologies like loops and delayed commitments in reasoning models. It proposes two lightweight control methods—FP16 planning and cycle rescue—which significantly restore reasoning accuracy while maintaining end-to-end speed.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-06-01T10:04:09.000Z
- 最近活动: 2026-06-02T03:31:05.380Z
- 热度: 124.5
- 关键词: 2-bit量化, 推理模型, 循环救援, FP16规划, 生成病理, Qwen3, 低比特推理, 模型量化
- 页面链接: https://www.zingnex.cn/en/forum/thread/2-qwen3-8b-17-2-74-2
- Canonical: https://www.zingnex.cn/forum/thread/2-qwen3-8b-17-2-74-2
- Markdown 来源: floors_fallback

---

## Introduction: Diagnosis and Recovery of Quantization Failure in 2-bit Reasoning Models

**Paper Core**: 2-bit quantization leads to generative pathologies such as loops and delayed commitments in reasoning models. Two lightweight control methods—FP16 planning and cycle rescue—are proposed, which boost Qwen3-8B's accuracy from 17.2% to 74.2% while maintaining end-to-end speed.

**Original Paper Information**:
- Author Team: Brain Lab Research
- Source: arXiv
- Title: Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery
- Link: http://arxiv.org/abs/2606.02011v1
- Code Repository: https://github.com/brain-lab-research/quantized-reasoning
- Publication Date: 2026-06-01

## Background: Cost Dilemma of Reasoning Models and the Temptation of Low-Bit Quantization

Large reasoning models (LRMs) like Qwen3 and DeepSeek-R1 improve accuracy by generating reasoning traces, but incur huge inference costs (more tokens, high computational overhead, increased latency).

Low-bit quantization (2/3/4-bit) is a popular direction to reduce costs, but aggressive 2-bit quantization not only reduces accuracy but may also fail to bring end-to-end acceleration—because the stability of the generation process is disrupted.

## Key Findings: Generative Pathologies Caused by 2-bit Quantization

2-bit quantization triggers **generative pathologies**, including:
1. **Repetitive Loops**: Getting stuck in identical/similar content loops and failing to progress;
2. **Budget Exhaustion**: Reaching the maximum length limit, leading to incomplete reasoning;
3. **Delayed Commitment**: Over-analyzing and wavering, adding unnecessary steps;
4. **Unclosed Reasoning Segments**: Structural chaos (e.g., mismatched parentheses).

Key Insight: The accuracy drop is closely related to these process-level failures, not the model being "dumber". Fixing generation stability can restore accuracy.

## Solutions: Two Lightweight Control Methods—FP16 Planning and Cycle Rescue

To address generative pathologies, two lightweight control methods are proposed:
**1. FP16 Planning**:
- Core: Use an FP16 model to generate a short reasoning outline, which is used as conditional input for the 2-bit model to fill in details;
- Advantages: Lightweight, effectively anchors the generation direction, and allows flexible adjustment of outline detail level.

**2. Cycle Rescue**:
- Core: Detect repetitive patterns, with intervention strategies including early submission (directly output when a reasonable answer is available) or FP16 fallback (switch to FP16 to continue generation);
- Detection Mechanisms: n-gram repetition detection, semantic similarity comparison, generation pattern analysis.

## Experimental Results: Significant Accuracy Recovery While Maintaining End-to-End Speed

**MATH-500 Benchmark Results**:
| Configuration | Qwen3-8B | Qwen3-32B |
|------|----------|-----------|
| 2-bit Baseline |17.2% |65.0% |
| + Cycle Rescue |74.2% |- |
| + Planning + Cycle Rescue |- |87.2% |

**Key Conclusions**:
- Qwen3-8B's accuracy increases by 57 percentage points, approaching FP16 performance;
- End-to-end speed maintained: The overhead of lightweight control is minimal, and the reduced number of tokens offsets additional computation, making overall speed better than the uncontrolled baseline;
- Consistent performance across tasks (math/common sense), with strong universality.

## Conclusions and Future Directions

**Conclusions**:
This study reveals that aggressive low-bit quantization disrupts the stability of the generation process in reasoning models (rather than just reducing quality). Through two lightweight methods—FP16 planning and cycle rescue—accuracy is significantly restored while maintaining speed.

**Future Directions**:
1. Adaptive Quantization: Dynamically adjust precision based on content;
2. Learning-based Control: Optimize control strategies using learning methods;
3. Hardware Collaboration: Combine with specific hardware characteristics;
4. Multi-model Collaboration: Collaborative reasoning among multiple quantized models.