Introduction: One-Sample Unsupervised Calibration Enables Reasoning Large Models to Gain "Self-Awareness"

This paper proposes a confidence calibration method for reasoning LLMs that requires no labeled data or repeated sampling. By training a lightweight confidence predictor via offline self-consistency distillation, it significantly improves model reliability. This method addresses the limitations of existing calibration techniques that rely on labeled data or increase inference overhead, providing support for deployment in high-risk scenarios.

Background: Reliability Dilemma of Reasoning Models and Limitations of Existing Methods

Large language models have improved reasoning capabilities, but they suffer from calibration bias: being overconfident in wrong answers or hesitant about correct ones, which restricts their application in high-risk scenarios. Confidence calibration is a core indicator of a model's "self-awareness", but existing methods have limitations:

1. Rely on labeled data, which is costly;
2. Require multiple samplings during inference (e.g., Self-Consistency), increasing latency and computational overhead. How to achieve effective calibration in one-sample inference scenarios has become a key issue.

Core Idea: Offline Distillation of Self-Consistency Signals for Unsupervised Calibration

The method consists of two phases: Offline Training Phase: Use a large number of unlabeled questions to sample the base model multiple times, generate multiple reasoning paths and answers, and calculate the consistency degree to construct a self-consistency proxy target (more identical answers mean higher reliability); train a lightweight predictor that takes a single reasoning path as input to learn to predict answer reliability (no manual labeling required). Deployment Phase: When the model generates a single answer, the predictor outputs a reliability estimate in real time, requiring only one forward pass with low latency.

Technical Details: From Self-Consistency Features to Robust Predictor Design

Key technologies:

1. Feature Transfer: Extract reasoning path features (length, certainty of intermediate steps, distribution of key nodes, generation probability characteristics, etc.), correlate these features with self-consistency scores, and learn statistical patterns;
2. Lightweight Predictor: Adopt MLP or small Transformer (1%-5% of the base model's parameter count), output a 0-1 calibration score after feature encoding, with the training target being to minimize the mean squared error with the proxy target;
3. Distributionally Robust Optimization: Offline sampling covers diverse tasks and difficulty levels to enhance generalization ability and handle distribution shifts.

Experimental Validation: Leading Performance Across Multiple Tasks and Models

Validated on 5 tasks (GSM8K, MATH, StrategyQA, HotpotQA, Natural Questions) and 9 models (7B-70B parameters, including Llama/Qwen/DeepSeek, etc.):

• Evaluation metrics (ECE, selective prediction accuracy, downstream decision-making) all outperform baselines (temperature scaling, Platt scaling, generation probability heuristics);
• Cross-domain testing (math training → QA application) maintains high accuracy in zero-shot transfer, while supervised methods show performance degradation;
• Selective prediction: Rejecting 30% of low-confidence questions increases the remaining accuracy by 8-15 percentage points.

Comparative Analysis: Advantages Over Traditional Methods

• vs Temperature Scaling: Non-intrusive, does not interfere with the generation process, and can be flexibly applied to any reasoning model;
• vs Self-Consistency: Maintains similar calibration accuracy while reducing inference overhead by 5-10 times (single generation + lightweight predictor);
• vs Supervised Methods: Unsupervised nature lowers application barriers, requires no labeled data, and is suitable for more scenarios.

Application Scenarios: Practical Value of High Efficiency and Low Cost

Applicable to:

1. Online Q&A Systems: Decide to display answers or transfer to humans based on confidence to improve experience and reduce risks;
2. Automatic Scoring Systems: Mark low-confidence answers for manual review to balance automation and quality;
3. Multi-Model Integration: Dynamically select the answer from the model with the highest confidence;
4. Continuous Learning: Guide active learning, prioritizing annotation of uncertain samples;
5. Interpretability: Gain insights into error-prone steps of the model through predictor features to assist optimization.

Limitations and Future Directions: Paths for Further Optimization

Limitations:

1. High computational overhead in the offline sampling phase (for ultra-large-scale models);
2. The predictor needs adjustment after the base model is fine-tuned or quantized;
3. Only evaluates confidence at the answer level, not involving intermediate reasoning steps.

Future Directions:

• Reduce the number of offline samplings;
• Enhance the predictor's robustness to changes in the base model;
• Refine calibration granularity to reasoning steps;
• Combine uncertainty quantification with interpretability to build more trustworthy AI systems.