ClinHallu: A Phased Benchmark for Hallucination Diagnosis in Medical Multimodal Large Models

ClinHallu is a phased hallucination diagnosis benchmark for medical multimodal large language models (MLLMs). Using 7,031 validation instances and structured reasoning tracking, it precisely locates the specific stages where hallucinations occur (visual recognition, knowledge recall, reasoning integration), providing a fine-grained testing tool for evaluating the credibility and safety of medical AI systems. It has been open-sourced.

Multimodal large language models have broad application prospects in the medical field, but the hallucination problem (generating seemingly reasonable but incorrect medical information) has serious consequences. Existing medical hallucination benchmarks only focus on identifying incorrect information and do not locate the reasoning stages where hallucinations occur (which link—visual understanding, knowledge recall, or reasoning integration—goes wrong).

The study found that hallucinations have diverse sources, and errors can occur in three stages: 1. Visual recognition stage (misidentifying lesions, anatomical structures, or imaging features); 2. Knowledge recall stage (biased or outdated medical knowledge); 3. Reasoning integration stage (logical leaps, causal confusion, etc.).

The core design of the ClinHallu benchmark includes: 1. Large-scale validation dataset (7,031 manually annotated instances); 2. Structured reasoning tracking (decomposed into tracking of three stages: visual recognition, knowledge recall, reasoning integration); 3. Phase replacement intervention mechanism (replacing the output of a specific stage with the correct answer to quantify the impact of each stage).

Using tracking supervised fine-tuning (with structured reasoning tracking as the supervision signal) can significantly reduce the hallucination rate of the model at each stage, improve the accuracy of the final answer, and enhance the interpretability and auditability of the reasoning process.

The practical significance of ClinHallu includes: 1. Improving diagnostic capabilities (precisely locating the source of hallucinations, facilitating targeted improvements or manual review); 2. Guiding model development (providing optimization directions: strengthening visual understanding, knowledge base, or reasoning capabilities); 3. Supporting regulatory compliance (meeting interpretability and safety requirements to facilitate clinical deployment).

ClinHallu has been open-sourced on GitHub (https://github.com/alibaba-damo-academy/ClinHallu), including a complete benchmark dataset, evaluation tools, and example code. Community contributions are welcome to improve it.

ClinHallu represents an important advancement in the field of medical AI evaluation. Through a phased diagnosis perspective, it provides fine-grained hallucination detection capabilities, offers new tools for understanding and improving the reasoning process of medical MLLMs, and helps build safer and more reliable clinical decision support systems.