ThinkSimplifier: Design and Implementation of a Reasoning Process Simplification System for Deep Thinking Models

ThinkSimplifier is a unified experimental framework designed to reproduce, evaluate, and integrate reasoning process simplification strategies for deep thinking language models. It targets the DeepSeek-R1-Distill-Qwen-7B model and provides configurable pipelines to compare various input-side (prompt optimization) and output-side (generation control) reasoning optimization methods, addressing the key challenge of balancing reasoning accuracy with efficiency (reducing token consumption and latency).

Recent deep thinking models (e.g., DeepSeek-R1, OpenAI o1/o3 series) excel in complex tasks like math reasoning via lengthy Chain-of-Thought (CoT) reasoning, but this leads to increased token usage, latency, and deployment costs. Traditional model compression methods (pruning, quantization, distillation) focus on parameters rather than structural optimization of reasoning processes. ThinkSimplifier fills this gap by providing a systematic platform for studying reasoning simplification strategies.

ThinkSimplifier uses a modular, config-driven design with key modules:

1. Unified Experiment Pipeline: End-to-end flow (data loading, model initialization, prompt building, generation control, answer extraction, metrics calculation) configurable via commands or files.
2. Input-side Strategies: Prompt optimizations like standard CoT, concise CoT, compressed CoT, draft chain (CoD), token limit prompts, budget-aware prompts.
3. Output-side Strategies: Early stopping mechanisms (answer consistency,阶段性CoT stop, confidence-driven stop, dynamic CoT allocation).
4. Answer Extraction & Evaluation: Dataset-specific format constraints (e.g., #### for GSM8K) and multi-metric evaluation (accuracy, avg tokens, speedup ratio).

ThinkSimplifier supports diverse math reasoning datasets:

• Basic: GSM8K, SVAMP, ASDiv.
• High-difficulty: AIME2024, AMC23, GSM-Hard.
• Comprehensive: MATH-500. It enables single and combined strategy experiments (input+output side). Evaluation metrics include accuracy, average token count, average time, compression ratio, speedup ratio, and outcome efficiency.

Key implementation details:

• Model Support: Defaults to DeepSeek-R1-Distill-Qwen-7B (7B params, accessible on consumer hardware) with extensible design for other models.
• Environment: Python3.10+, PyTorch, Transformers, Datasets. Optional DeepSeek API integration for answer extraction.
• Reproducibility: Fixed random seeds, temperature, sampling; timestamped experiment outputs for traceability.

ThinkSimplifier's value includes:

• Education: Ideal for undergrad projects/theses (modular design, easy to use).
• Open Source Evaluation: Standardized platform for comparing open-source deep thinking models.
• Strategy Integration: Facilitates research on combined input/output strategies.
• Industrial: Guides enterprises to choose optimal simplification strategies for production deployment (balance accuracy and cost).

Future Plans: Support more models, expand beyond math datasets, add visualization tools, automate ablation studies, improve robustness for symbolic expressions. Conclusion: ThinkSimplifier fills the gap of a unified framework for reasoning simplification. By combining input-side prompt optimization and output-side generation control, it helps explore the accuracy-efficiency tradeoff, critical for reducing deployment costs and improving user experience of deep thinking models.