Deep Dive into Sampling Strategies for Large Model Reasoning: Why Parallel Sampling Outperforms Sequential Sampling

The latest study Understanding Performance Gap Between Parallel and Sequential Sampling in Large Reasoning Models delves into the performance differences between parallel and sequential sampling in large reasoning models, revealing that insufficient exploration is the main reason for the poor performance of sequential sampling. The study involves models such as Qwen3, DeepSeek-R1 distilled model, and Gemini 2.5, covering tasks like mathematical reasoning and code generation.

Large reasoning models struggle to obtain optimal solutions with a single sampling in complex tasks, requiring multiple samplings. Sampling strategies are divided into parallel and sequential: parallel sampling generates multiple independent reasoning paths simultaneously and selects the best via an aggregator; sequential sampling generates iteratively, using previous answers as context to continue generation. Theoretically, it should be stronger, but in practice, its performance is worse than parallel sampling.

To explain the performance gap, the research team proposed and verified three hypotheses: 1. The advantage of parallel sampling comes from the aggregation operation; 2. Sequential sampling's quality is affected by context length limitations; 3. The conditional dependence of sequential sampling limits the exploration space. Experiments cover multiple model families and task domains.

Experimental results show that aggregation operations and context length are not the main causes; the difference in exploration ability is the key. Sequential sampling tends to fine-tune based on existing answers, limiting the discovery of new problem-solving paths; parallel sampling generates independently, maintains a complete exploration space, and is more likely to find optimal solutions.

Conclusion: Maintaining exploration diversity is more important than sequential incremental improvement. Implications: Prioritize parallel sampling; if using sequential sampling, design mechanisms to prevent premature convergence. The study reminds us that theoretical intuition in AI systems may not align with reality, and empirical research is crucial to building more reliable and efficient intelligent systems.