Less is More: ViTAS Revolutionizes Medical Image Report Summarization via Selective Visual Attention

ViTAS proposes a revolutionary multimodal radiology report summarization method. By selectively focusing on pathology-related visual regions instead of the entire image, it achieves state-of-the-art (SOTA) performance on the MIMIC-CXR benchmark (BLEU-4 at 29.25%, ROUGE-L at 69.83%), validating the superiority of the "less is more" visual strategy in medical image analysis.

Automatic medical image report generation is a challenging task in medical AI. Traditional methods rely on the text modality; after the rise of multimodal models, attempts have been made to integrate visual information, but their performance often lags behind pure text baselines and is also disturbed by visual noise. This raises questions: Is more visual input better? Is multimodal still valuable when the text already contains image descriptions?

The study verifies a counterintuitive hypothesis through ablation experiments: selectively focusing on pathology-related visual regions instead of the entire image can significantly improve performance. The entire image contains a large number of irrelevant anatomical structures (such as normal lung tissue), which easily introduces noise and distracts attention; focusing on pathological regions allows precise understanding of lesions, similar to how radiologists read images.

ViTAS includes four processing stages: 1. Intelligent region segmentation (using MedSAM2 to adaptively segment lung regions, integrated with guidance optimization); 2. Multi-view bidirectional cross-attention (fusing complementary information from different views); 3. Shapley value-guided adaptive clustering (quantifying the contribution of regions and selecting high-value regions); 4. Hierarchical visual tokenization (converting to sequences suitable for ViT, fusing with text features to generate impressions).

ViTAS achieves 29.25% BLEU-4 and 69.83% ROUGE-L on MIMIC-CXR, significantly outperforming existing methods. High scores in automatic evaluation metrics mean the generated content is highly consistent with expert-written reports at both lexical and semantic levels; qualitative analysis and expert evaluation show that factual consistency is better than baselines, with the highest human scores, indicating potential for clinical application.

ViTAS provides the following insights: 1. The quality of visual input is more important than quantity; 2. Combining domain knowledge (anatomical segmentation, multi-view fusion) with deep learning improves reliability; 3. Interpretability tools like Shapley values enhance performance and decision transparency. In the future, it can be extended to modalities such as CT/MRI and tasks like lesion detection and disease grading; the "less is more" strategy may inspire other computer vision scenarios.

ViTAS is an important milestone in medical imaging AI, setting a new record for report summarization performance and challenging the traditional understanding of multimodal learning. It reminds us that progress in medical AI lies not only in scale expansion but also in strategy optimization; precise and accurate visual understanding is the key path to reliable and practical medical AI.