# When Does Built-in Thinking Help or Harm? A Study on Constraint-Level Error Transfer in Instruction Following

> The study reveals the dual nature of large reasoning models in instruction-following tasks—thinking mode benefits planning-type constraints but harms precision-type constraints, with 10-20% of prompts showing reversed results when switching between thinking modes.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-06-08T15:45:04.000Z
- 最近活动: 2026-06-09T05:24:53.396Z
- 热度: 135.3
- 关键词: 大推理模型, 指令遵循, IFEval, 思考模式, 约束分类, Qwen3, 模型评估
- 页面链接: https://www.zingnex.cn/en/forum/thread/llm-arxiv-2606-09662v1
- Canonical: https://www.zingnex.cn/forum/thread/llm-arxiv-2606-09662v1
- Markdown 来源: floors_fallback

---

## [Introduction] The Dual Nature of Built-in Thinking: A Study on Constraint-Level Error Transfer in Instruction Following

The study reveals the dual nature of large reasoning models in instruction-following tasks—thinking mode benefits planning-type constraints but harms precision-type constraints, with 10-20% of prompts showing reversed results when switching between thinking modes. This research targets the Qwen3 series and Hunyuan series models, analyzing the impact of built-in thinking on different types of constraints through controlled experiments, providing important insights for model design and application.

## Research Background: Unsolved Mysteries of Instruction Following and Test Benchmarks

The built-in thinking ability of large reasoning models (e.g., OpenAI o1/o3, DeepSeek-R1) performs well in math/programming tasks, but its impact on instruction following remains unclear. IFEval is an authoritative benchmark for testing models' compliance with complex instructions, including verifiable constraints (format, length, keywords, etc.). The challenge faced by the study is how to disentangle the confounding effects of "thinking" from factors like model size and training data.

## Research Methods: Same-Weight Control and Cross-Family Validation

The Alibaba Qwen3 series (1.7B-32B) was selected because its "thinking on/off" modes use the same weights, allowing precise isolation of the thinking variable; the Tencent Hunyuan series (four sizes) was included for cross-family validation to ensure the generality of the findings.

## Key Findings: Error Pattern Transfer and Constraint Type Differentiation

The overall pass rate changed slightly (Qwen3 decreased by 0.55-3.52 percentage points), but 10-20% of prompts showed reversed results, indicating that thinking changes the error distribution. Constraints are divided into two categories: planning-type (global counting, structural requirements, etc.) improved under thinking mode; precision-type (exact format, character-level constraints, etc.) consistently deteriorated. Although the overall trend of the Hunyuan model was opposite, the symbolic pattern of constraint classification was consistent.

## Mechanism Exploration: Deep Reasons for the Impact of Thinking

Thinking mode changes the length of answers; after controlling for length, the loss of precision-type constraints decreased but residual penalties still existed. Correlation analysis of thinking traces showed: positive correlation in neutral mode (r≈0.15), almost no correlation in planning mode (r≈0.02, with an execution gap), and negative correlation in precision mode (r≈-0.05). Activation repair experiments indicated that flipped instances of precision-type constraints were easier to fix (32-58%), while planning-type ones were harder (14-40%).

## Practical Implications and Future Directions

**Practical Implications**: Dynamically adjust thinking mode (enable for planning-intensive tasks, disable for precision-intensive tasks); design hybrid reasoning architectures (task routers, adaptive depth, constraint-aware reasoning). **Limitations**: Single benchmark (IFEval), limited model scope, need for deeper mechanism explanations. **Future Directions**: Multi-benchmark validation, theoretical modeling, intervention strategy development, training optimization.

## Conclusion: The Selective Value of Thinking

Built-in thinking has a highly selective impact on instruction following, being a double-edged sword—it benefits planning-type constraints but harms precision-type constraints. Simply enabling thinking is not an optimal strategy; reasoning modes need to be adjusted according to task structure. This finding has far-reaching implications for model design and application, as true intelligence lies in "knowing when to think and when to act directly."