Multimodal Fusion for Predicting Respiratory Failure: How Chest X-rays Outperform Traditional EHR Signals

Original Authors and Source

• Original Author/Maintainer: arXiv authors
• Source Platform: arXiv
• Original Title: Prospective evaluation of multimodal respiratory failure prediction: Do chest X-rays improve performance beyond EHR signals?
• Original Link: http://arxiv.org/abs/2605.26255v1
• Source Publication/Update Time: 2026-05-25T18:25:47Z

This article is a prospective study aimed at exploring the fusion of chest X-ray (CXR) images with electronic health record (EHR) time-series data to predict whether ICU patients need mechanical ventilation within 24 hours. The proposed gated multimodal framework significantly outperforms EHR-only models in AUROC (0.860 vs. 0.752), demonstrating the value of foundational medical imaging models in clinical prediction.

Core Challenges in ICU Respiratory Failure Prediction

In the intensive care unit (ICU), early identification of respiratory failure is directly related to patient survival. Traditional monitoring relies on physiological indicators in EHR (heart rate, blood pressure, oxygen saturation, etc.), but these signals cannot fully reflect the pathophysiological changes in the lungs. Chest X-rays, as a routine ICU examination, can provide key information such as pulmonary infiltration and pulmonary edema, but they are difficult to effectively fuse with dynamic EHR data—this is a key pain point for clinical AI.

Key Components of the Gated Multimodal Framework

The innovative framework designed by the research team includes three core parts:

1. EHR Encoder: Processes structured time-series physiological data to capture the dynamic evolution of patient status;
2. CXR Encoder: Extracts high-dimensional representations based on foundational medical imaging models such as REMEDIS and MedInsight;
3. Gating Module: Adaptively adjusts the contribution weight of image features according to clinical context, simulating human decision-making logic (e.g., less reliance on images when EHR data is clear, more use of images when EHR data is ambiguous).

Prospective Evaluation Results

Comparison with EHR-only Models

Model	AUROC	Features
Vent.io (EHR-only)	0.752	Uses only time-series physiological data
Gated Multimodal (REMEDIS)	0.860	Fuses EHR + CXR
Gated Multimodal (MedInsight)	0.858	Fuses EHR + CXR

Comparison with Physician Predictions

The multimodal framework significantly improved sensitivity (capturing more patients in need of intervention) while maintaining specificity. It also increased specificity and positive predictive value (PPV), reducing the clinical burden of false positives.

Complementarity Between Images and EHR

EHR has three major limitations:

• Lag: Physiological abnormalities occur later than pulmonary pathophysiological changes;
• Non-specificity: The same sign corresponds to multiple causes;
• Noise: Device data is subject to interference.

In contrast, CXR can directly visualize early pulmonary pathology (e.g., pulmonary consolidation, edema) before physiological indicators fluctuate. The gating mechanism avoids redundancy/conflicts from simple fusion, allowing the model to adaptively utilize information.

Practical Value of Foundational Medical Imaging Models

This study demonstrates the transfer potential of pre-trained models like REMEDIS in clinical workflows. Future clinical decision support systems can integrate multimodal information to provide interpretable warnings (e.g., "75% risk based on pulmonary edema signs") and serve as intelligent assistants for doctors.

Study Limitations and Next Steps

Limitations

• Not all patients can get timely CXR;
• Differences in image quality, body position, etc., affect model stability;
• Larger-scale randomized controlled trials are needed for validation.

Future Directions

• Explore more frequent image acquisition methods such as portable ultrasound;
• Integrate more modalities like laboratory results and ventilator parameters;
• Develop multimodal frameworks for predicting other organ failures.