Inference Model Inference-Time Computation Optimization: Strategies to Maximize Accuracy Under Fixed Budget

This article deeply explores how to maximize the accuracy of inference models on math test sets using cutting-edge inference-time computation strategies such as majority voting and PRM-guided beam search under a fixed computation budget. The study systematically compares the performance of various methods and provides key guidance for the practical deployment of inference models.

Large-scale inference models (e.g., GPT-4, Claude) excel at complex mathematical problems but have high computational costs. In practical applications, resources cannot be expanded indefinitely, so we focus on optimizing resource allocation through "inference-time computation" strategies. The study selects the MATH test set (high-difficulty math competition problems, the gold standard for reasoning ability), with the core question: Which strategy can maximize problem-solving accuracy under a fixed budget?

The study evaluates four mainstream strategies:

1. Majority Voting: Generate multiple independent solutions, vote to select the most frequent answer; simple to implement but treats all solutions equally.
2. Naive Optimal N-Select (PRM): Generate N candidates, use PRM (Process Reward Model) to score and select the highest; finely identifies high-quality reasoning paths.
3. Weighted Optimal N-Select (PRM): Introduce a weight mechanism to balance relative quality differences among candidates, enhancing robustness for complex problems.
4. PRM-Guided Beam Search: Maintain a beam of K candidates at each step; PRM scoring retains high-scoring paths for expansion, systematically exploring the solution space.

Under fixed budgets, PRM-based strategies are generally superior to majority voting (process-level feedback improves reasoning quality); beam search performs prominently in medium budgets (dynamic resource allocation reduces waste); applicable scenarios for different budgets: use majority voting with a small number of samples for extremely limited budgets, and beam search to explore deep reasoning patterns for sufficient budgets.

Enterprise-level applications: Choosing the right strategy can reduce costs and improve efficiency (e.g., using beam search in online math tutoring to ensure speed and quality); research directions: more efficient PRM design, combining inference-time computation with fine-tuning; methods can be extended to code generation, scientific reasoning, and other fields.

Key components: High-quality PRM (evaluates the rationality of reasoning steps), efficient sampling mechanism (generates diverse candidates), search algorithms that balance exploration and exploitation; strategies can be combined (e.g., beam search to generate candidates + majority voting for decision-making).

Inference-time computation optimization is an important direction for improving LLM reasoning capabilities; smart use of computing power is more valuable than piling up computing resources. This study provides a practical guide, and future AI systems will demonstrate stronger reasoning capabilities under more efficient computing models.