Economic Analysis of Large Model Distillation Strategies: Trade-off Between Reasoning-Trace Distillation and Answer-Only Distillation

This study systematically compares the economic efficiency and performance of reasoning-trace distillation and answer-only distillation in Transformer language models, aiming to provide a quantitative decision-making basis for model compression and edge deployment. The two strategies differ significantly in training cost, inference performance, and final effect. This project constructs a decision framework through systematic evaluation to help practitioners balance and choose.

While large language models have improved capabilities, their massive parameter count leads to a sharp increase in deployment costs. As an important compression technique, model distillation transfers knowledge from large models to small models to adapt to resource-constrained environments. However, the choice of distillation strategy is unclear: traditional answer-only distillation uses only final output for supervision, while emerging reasoning-trace distillation retains intermediate steps. The two differ significantly in training cost, inference performance, and effect. This study provides a quantitative decision framework through economic and performance evaluation.

Answer-Only Distillation

A classic distillation paradigm where the teacher model generates final answers, and the student model learns the direct input-output mapping. Advantages: Simple data preparation, fast training speed; Limitations: Students cannot understand the reasoning process.

Reasoning-Trace Distillation

With the popularization of chain-of-thought technology, retaining the reasoning process improves interpretability and generalization ability. The teacher outputs complete thinking steps, and the student learns the full mapping of problem→reasoning→answer. Advantages: Inherits the teacher's reasoning ability, performs well on complex tasks; Limitations: More training data, longer sequence processing, high computational overhead.

Training Cost Analysis

Reasoning-trace distillation processes longer sequences (5-10 times that of answers), leading to high memory usage (batch size limited), long training time (quadratic complexity of attention calculation), and high data annotation cost (requiring stronger teacher model APIs).

Inference Efficiency Comparison

Models trained with reasoning-trace distillation can accurately self-correct on complex problems, reducing the need for repeated queries.

Total Cost of Ownership (TCO) Model

Comprehensive trade-off between training, inference costs, and accuracy: In high-frequency call scenarios, the initial high investment of reasoning-trace distillation can be offset by long-term efficiency improvements; for low-frequency/simple tasks, answer-only distillation is more economical.

Task Complexity and Strategy Matching

Complex tasks (mathematical reasoning, code generation): Reasoning-trace distillation improves accuracy by 15-25%; Simple tasks (sentiment analysis, text classification): The performance gap between the two is small.

Impact of Model Scale

Extremely small student models (<1B parameters): Answer-only distillation is better (difficult to learn complex reasoning); Medium-scale models (3B-7B parameters): The advantages of reasoning-trace distillation become apparent.

Domain Transfer Capability

Reasoning-trace distillation models, having learned general reasoning patterns, perform more robustly on new domain data and are suitable for scenarios with rapid business changes.

Scenarios for Choosing Answer-Only Distillation

• Simple tasks that do not require complex reasoning
• Limited training budget requiring rapid iteration
• Extremely high requirements for inference latency
• Mainly high-frequency simple queries

Scenarios for Choosing Reasoning-Trace Distillation

• Multi-step logical reasoning tasks (mathematics, code, planning)
• Need for model interpretability
• Need for self-correction/reflection capabilities
• Long-term operation scenarios (training costs can be amortized)

Hybrid Strategy Possibility

Two-stage approach: Use answer-only distillation for rapid convergence in the initial stage, then fine-tune with reasoning-trace distillation in the later stage to balance cost and performance.

Industry Impact

• Edge AI Deployment: Provides quantitative guidance for resource-constrained environments such as smartphones and IoT.
• Model-as-a-Service (MaaS) Optimization: Helps vendors optimize pricing and resource allocation; reasoning-trace models create value through high accuracy and low retry rates.

Future Research Directions

• Adaptive distillation: Dynamically select strategies
• Hierarchical distillation: Use different targets for different components
• Multi-teacher distillation: Integrate the advantages of two teacher models