DECAT: A Framework for Assessing Biological Authenticity in Multimodal Medical AI Diagnosis

The research team has launched the DECAT (Diagnostic Evaluation of Cross-modal Alignment and Trustworthiness) framework, which aims to diagnose whether multimodal oncology AI models truly learn cross-modal shared biological features rather than relying on spurious correlations. This model-agnostic post-hoc evaluation tool has been validated on real TCGA data, helping to enhance the clinical credibility and safety of AI systems.

In the field of oncology AI, although multimodal models can integrate pathological images, genomic data, etc., to achieve high prediction accuracy, high accuracy does not necessarily mean that the model understands the underlying biological mechanisms. A model may learn real biological features shared across modalities, limited features from a single modality, or only capture non-causal confounding factors related to outcomes (such as differences in hospital equipment, population characteristics). While these three scenarios have similar predictive performance, their clinical generalization capabilities are vastly different.

DECAT is a post-hoc evaluation framework that acts directly on model representations without needing to know specific confounding factors. It classifies multimodal representations into four diagnostic scenarios: 1. Cross-modal shared biology (ideal case, multimodality collaboratively captures the same biological reality); 2. Single-modality limited biology (real features but limited to a single modality); 3. Confounding factor-driven (dangerous, relying on spurious correlations); 4. Uncertain (conservative result when evidence is insufficient). The evaluation is based on five zero-reference metrics, covering dimensions such as representation independence and cross-modal alignment consistency.

The team validated the effectiveness of DECAT through large-scale experiments: 1. Over 2500 multimodal representations were trained on synthetic data, and DECAT could accurately identify the four scenarios; 2. Evaluation on real TCGA data (multimodal data of 8979 patients) showed that CLIP-like entangled models performed perfectly in detecting shared biology, but real foundation model embeddings often falsely claimed the existence of shared biology when there was none; 3. Increased strength of confounding factors raises the misdiagnosis rate, and larger cohorts and stronger models may lead to overconfidence.

DECAT reveals the limitations of the traditional metric AUROC—it cannot detect confounding factors. A model may have excellent AUROC but rely on spurious correlations. DECAT can detect hidden issues without confounding factor labels, identifying reliability risks before deployment. Evaluation of five pathological foundation models shows that they have limitations when there is no paired RNA data, requiring careful validation.

DECAT brings three major contributions to medical AI evaluation: 1. Flexibility of post-hoc evaluation, which can be applied to trained models without modifying the architecture; 2. Confounding factor irrelevance, which can detect unknown or unexpected confounding factors; 3. Statistical rigor, providing reliable evaluation through zero-reference metrics and rule-based decisions, with conservative handling of uncertain results.

DECAT has the following limitations and improvement directions: 1. Statistical power issue—diagnostic reliability is limited when data volume is insufficient; future exploration of Bayesian methods to quantify uncertainty is needed; 2. Currently, it mainly handles single confounding factors and needs to be extended to scenarios with interactions of multiple confounding factors; 3. Integrate causal inference methods to deepen understanding of biological mechanisms.

The launch of the DECAT framework is an important progress in the field of medical AI evaluation. It provides a systematic method to diagnose the nature of multimodal representations, distinguish between real biological learning and spurious correlations, and help enhance the clinical credibility and safety of AI systems. In today's era of AI medical popularization, such rigorous evaluation tools are of great value for ensuring patient safety and promoting the healthy development of technology.