A New Reasoning Method Based on Decision Point Sampling: The Entropy-Cut Metropolis-Hastings Algorithm

This article introduces the Entropy-Cut Metropolis-Hastings (Entropy-Cut MH) algorithm, a new reasoning method based on decision point sampling. The core innovation lies in using next-token entropy to identify key decision points, enabling more efficient power distribution sampling. This algorithm outperforms baseline methods and RL-trained models on multiple reasoning benchmarks, challenging the traditional notion that "reasoning must be acquired through RL training" and revealing that pre-trained models already contain strong reasoning capabilities, providing a new paradigm for optimizing reasoning efficiency. Source: arXiv paper "Reasoning with Sampling: Cutting at Decision Points" (2026-05-28, link: http://arxiv.org/abs/2605.30327v1)

Current state-of-the-art reasoning models mostly acquire their capabilities through reinforcement learning (RL) post-training, but RL training requires large computational resources, carefully curated datasets, and complex reward mechanisms. Recent studies have found that by "sharpening" the base model's distribution (sampling from the power distribution p(x)^α, α>1), reasoning capabilities comparable to RL models can be unlocked without RL training, curated datasets, or verifiers. This indicates that reasoning capabilities may exist more in pre-trained models rather than having to be injected through RL.

The key obstacle to the practical application of power distribution sampling is efficient sampling. Existing methods use the Metropolis-Hastings framework, exploring paths by uniformly randomly selecting cut points to resample suffixes, but there are defects: reasoning trajectories contain a small number of key decision points (3-5) and a large number of local details (hundreds of tokens). Uniform cutting often falls on details, only rewriting wording/computational details without changing the reasoning strategy, leading to low sampling efficiency.

The core of the Entropy-Cut MH algorithm is to use next-token entropy to identify decision points: when the model faces important decisions, the prediction distribution is scattered (high entropy), and when performing deterministic calculations, it is concentrated (low entropy). Algorithm flow: 1. Calculate the next-token entropy at each position of the current trajectory; 2. Detect local entropy peaks/jump points; 3. Select the cut position with a probability positively correlated with entropy; 4. Accept new samples according to the MH criterion.

Theoretical Analysis: Simplified models prove that the Entropy-Cut mixing time only grows with the number of decision points (far less than the number of tokens), while uniform cutting grows with the number of tokens, achieving an order-of-magnitude acceleration. Experimental Verification: On benchmarks such as MATH500, HumanEval, GPQA Diamond, and AIME26, Entropy-Cut outperforms the uniform cutting baseline under the same sampling budget, matches or exceeds RL models, and requires fewer sampling steps. Ablation experiments prove the effectiveness of the entropy signal—other signals have poor effects, and MH correction is indispensable.

Deep Significance: Sampling strategies can stimulate reasoning capabilities, indicating that pre-trained models already contain reasoning capabilities—RL may only guide/stabilize rather than construct them; sampling can serve as an alternative paradigm to RL training; it promotes resource allocation for "test-time computation" (intelligent search during reasoning instead of RL optimization during training). Application Prospects: Zero-training-cost reasoning enhancement; synergy with RL models to improve performance; only requires logits output, easy to implement in open source.

Limitations: High entropy may be due to model confusion rather than decision points; low entropy may be due to certainty rather than details; decision point dependencies are complex in complex reasoning tasks. Future Directions: Develop more refined decision point identification methods; explore efficient sampling for multi-step complex reasoning scenarios; combine technologies such as verifiers and process reward models to improve efficiency.

The Entropy-Cut MH algorithm is an important advancement in the field of reasoning sampling. It achieves efficient power distribution sampling through entropy-based decision point identification, showing advantages in both theory and experiments. It challenges traditional concepts, provides valuable tools and ideological frameworks for reasoning efficiency, cost optimization, and model capability mining, and has important reference significance for researchers and engineers.