Mitigating Visual Hallucinations in Multimodal Systems via Retrieval-Enhanced Reliability-Aware Reasoning

This paper proposes a retrieval-enhanced reliability-aware reasoning framework aimed at addressing the visual hallucination problem in multimodal systems. The core idea is to construct an external visual evidence database, combine multiple reliability metrics to evaluate prediction credibility, and dynamically adjust output strategies through a decision gating mechanism. This method can effectively improve prediction accuracy and reduce hallucination rates without retraining large multimodal models, providing more reliable solutions for key scenarios such as medical imaging and autonomous driving.

Research Background and Motivation

Multimodal Large Language Models (MLLMs) are powerful in visual-language understanding and response generation, but they tend to produce overconfident hallucinatory outputs when faced with weak, ambiguous, or semantically inconsistent visual evidence, which seriously affects the reliability of key scenarios such as medical image analysis and autonomous driving. Existing solutions mostly focus on improving representation alignment or retrieval-augmented generation, but lack mechanisms to quantify instance-level prediction reliability and identify erroneous outputs, making it difficult for systems to know their own confidence and avoid answering when evidence is insufficient.

Core Method: Reliability-Aware Reasoning Framework

The framework includes four key components:

1. External Visual Evidence Database: Built using a pre-trained visual embedding model, finds similar reference samples via nearest-neighbor retrieval, and does not rely on task-specific training data.
2. Multiple Reliability Metrics: Fuses similarity strength, category support consistency, evidence boundary, and entropy-based uncertainty to generate a comprehensive reliability score.
3. Decision Gating Mechanism: Adopts three-level strategies based on scores: accept prediction (high threshold), cautious response (medium), avoid/fallback (low threshold).
4. Multimodal Response Generation Layer: Conditionally generates responses with confidence information based on the decision gate output.

Experimental Validation and Result Analysis

Evaluations on the ImageNet-100 dataset show:

• Accuracy Improvement: The accuracy of accepted predictions increased from 85.84% to 88.88%, while coverage remained at 89.04%.
• Hallucination Rate Reduction: The false acceptance rate decreased from 14.16% to 11.12%, reducing erroneous predictions by approximately 21%.
• Method Advantages: No need to retrain large models; dynamically adjusts behavior during inference, suitable for improving deployed models.

Technical Insights and Application Value

Technical Insights:

• Retrieval enhancement can be used for verification (not just generation);
• Multi-metric fusion is more robust than single metrics;
• Selective response strategies effectively reduce hallucinations. Application Scenarios:
• Medical image diagnosis: Mark uncertain cases for manual review;
• Autonomous driving: Trigger safety fallback when uncertain;
• Content moderation: Reduce false positives and false negatives;
• Intelligent customer service: Request clarification for ambiguous inputs.

Limitations and Future Directions

Limitations:

• Building and maintaining the external evidence database requires additional resources, and retrieval quality affects evaluation accuracy;
• Decision thresholds need to balance coverage and accuracy, requiring different configurations for different scenarios. Future Directions:
• Extend to more modalities such as audio and video;
• Explore online learning to dynamically update the evidence database;
• Combine model fine-tuning to further improve performance.

Conclusion

This study provides a practical and efficient solution to improve the reliability of multimodal systems. Through retrieval-enhanced reliability-aware reasoning, it can significantly reduce visual hallucinations and improve prediction accuracy without retraining, laying the foundation for building trustworthy, transparent AI systems and their safe deployment in the real world.