CERD: An Interpretable Multimodal Medical Diagnosis Framework for Incomplete Modalities

CERD proposes an interpretable multimodal diagnosis framework for handling missing modality data. Through conditional evidence reconstruction and logit-level attribution decomposition, it can reconstruct missing representations when modalities are incomplete, decompose diagnostic evidence into cross-modal shared confirmation and modality-specific clues, and achieve excellent performance on the ADNI dataset.

Neurodegenerative diseases (e.g., Alzheimer's disease) are multifactorial. Single-modality diagnosis struggles to capture comprehensive features, making multimodal modeling a trend. However, modality coverage is often incomplete in practice. Existing methods are fragile when facing missing modalities, rely on group-level priors, lack interpretability, and fail to meet clinical needs.

Conditional Evidence Reconstruction

Reconstruct missing modality representations based on the subject's existing data, with advantages of individualization, cross-modal dependency modeling, and preservation of diagnostic value.

Evidence Decomposition and Attribution

Through logit-level attribution, decompose diagnostic evidence into cross-modal shared confirmation (consistent support from multiple modalities) and modality-specific clues (unique information from a single modality), providing structured explanations.

Representation Learning

Use deep learning to learn representations of each modality; the encoder can handle incomplete inputs.

Conditional Reconstruction Mechanism

Utilize conditional generative models to infer reasonable representations of missing modalities based on observed modalities.

Logit-level Attribution

Analyze the contribution of each evidence source to diagnostic probability at the logit level, which is closer to the decision-making process.

In the evaluation on the ADNI dataset (containing multimodal neurodegenerative disease data), CERD significantly outperforms baseline methods, and the conditional reconstruction strategy effectively utilizes available information; its evidence attribution is structured and aligned with clinical knowledge, consistent with the cognition of medical experts.

• Improve diagnostic accessibility: Reliable support can be obtained in resource-constrained environments without a complete set of examinations.
• Enhance doctor trust: Transparent evidence decomposition helps understand the basis of AI judgments.
• Support personalized medicine: Conditional reconstruction is based on the patient's unique data, aligning with the concept of precision medicine.

• Reconstruction quality assessment: In-depth research on uncertainty quantification of reconstructed representations is needed.
• Rare modality missing: The reliability of reconstruction when key modalities are missing needs additional verification.
• Cross-disease generalization: Its applicability to other neurodegenerative diseases needs to be verified.

CERD is an important advancement in the field of multimodal diagnosis with incomplete modalities, balancing high diagnostic accuracy and structured interpretability. ADNI experiments verify its clinical potential. As multimodal medical AI develops, such frameworks will play an important role.