NeurIPS 2025 Study: Reasoning-Based Bias Detector Turns Any Large Language Model Into a Reliable Judge

A joint research team from the National University of Singapore and Tsinghua University proposed an innovative reasoning-based bias detector. By having large language models (LLMs) explicitly generate a reasoning process before making judgments and self-detect biases, their reliability as judges is significantly improved. This work has been accepted by NeurIPS 2025.

LLMs are increasingly used as judges (e.g., evaluating text quality, model output performance, etc.), but they have position bias (preferring options in specific positions), length bias (preferring longer answers), and self-enhancement bias (giving higher scores to themselves or similar models), which seriously affect reliability and limit deployment in key scenarios.

The core innovation is the reasoning-based bias detector, which does not directly eliminate biases but instead allows LLMs to first generate a complete reasoning process, then detect and quantify biases based on this process. The technical implementation is divided into two stages: the reasoning generation stage (generating detailed judgment reasoning) and the bias detection stage (analyzing the reasoning to get a bias score and deciding whether to accept or re-judge). The method is model-agnostic and applicable to any LLM (e.g., GPT-4, Claude, open-source models).

Validated effective on multiple benchmark datasets: After debiasing, the consistency between LLM judgments and human judgments is significantly improved, and judgment stability is enhanced. Especially in handling position bias, it performs excellently, greatly reducing sensitivity to option order, making results more stable and reliable.

It provides a practical tool for building reliable LLM judgment systems, promoting the application of automated evaluation in more key scenarios; reveals the value of reasoning processes in bias detection; provides an out-of-the-box solution for AI developers, applicable to scenarios such as model comparison, quality assessment, and automated evaluation processes.

This study opens up a new direction for solving the reliability problem of LLM judgments. It has significant effects and wide applicability, and is expected to play an important role in future AI evaluation practices. As LLMs are applied in key decision-making scenarios, ensuring the fairness and reliability of judgments becomes increasingly important, and this study is a solid step forward.