SinkProbe: A New Method for Detecting Hallucinations in Large Language Models Using Attention Sinks

A research team from Wroclaw University of Science and Technology in Poland proposed the SinkProbe method, which detects hallucinatory content by analyzing internal attention sinks of large language models. This method does not require external references, uses only statistical features of attention matrices to achieve efficient detection, and has performed excellently on multiple models and datasets. The related paper will be published at ICML 2026.

Large language models (LLMs) often generate hallucinatory content that seems reasonable but is incorrect, seriously affecting the reliability of critical fields such as healthcare and law. Traditional detection methods rely on external knowledge bases (with limited coverage) or manual annotation (high cost), so there is a need to explore detection solutions based on the model's internal state.

The core of SinkProbe is to use attention sinks (specific tokens that the model continuously focuses on) as internal signals, without requiring external references or additional training data. Compared to baseline methods like AttentionScore, it adopts a more concise feature extraction strategy: calculating the order statistics of attention head sink scores, balancing semantic information and computational efficiency.

The project is based on Python 3.13 and the uv package management tool, with a modular design:

• Feature extraction module (hallucinations/): Extracts features such as sink scores and back-looking ratios from attention matrices;
• Data pipeline (scripts/dataset/): Supports automatic processing of 7 mainstream question-answering datasets;
• Evaluation framework (scripts/eval/): Integrates n-gram metrics and GPT-4.1 as evaluation criteria;
• Probe training (scripts/probes/): Implements training and cross-validation for SinkProbe and baseline methods.

Tests covered 4 open-source models (Llama3.2-3B, Llama3.1-8B, Phi3.5 Mini, Mistral NeMo) and 7 datasets, and SinkProbe's performance reached or exceeded the existing state-of-the-art methods. Its computational overhead is significantly lower than methods requiring full attention transformation, making it suitable for latency-sensitive production environments.

• For developers: Can be integrated into LLM services as a real-time detection layer to mark high-risk outputs or trigger manual review;
• For researchers: Provides a complete reproducible workflow (7 idempotent stages, recoverable from interruptions), lowering the threshold for reproducing paper results.

Limitations: Only applicable to Transformer decoder models; not validated on architectures like RWKV/Mamba; needs to combine other signals in scenarios such as open-ended generation and code generation. Future directions: Multimodal hallucination detection, lightweight online algorithms, combining internal signals with external knowledge retrieval.

SinkProbe advances LLM interpretability research. By exploring the internal structure of the attention mechanism, it provides efficient and reliable hallucination detection signals, with both theoretical value and practical application significance, offering technical support for the safe deployment of LLMs in critical fields.