Multimodal Machine Learning for Predicting Heart Disease Severity: A Fusion Analysis of Clinical Data and ECG

This article introduces a multimodal machine learning framework that integrates clinical indicators and electrocardiogram (ECG) signals. Using feature engineering and random forest ensemble methods, it achieves accurate grading prediction of five heart disease severity levels (healthy, mild lesion, moderate lesion, severe lesion, critical), demonstrating the value of multi-source data fusion in medical prediction.

Cardiovascular disease is the leading cause of death globally. Early detection and accurate grading are crucial for improving prognosis. Traditional diagnosis relies on expert judgment, which is time-consuming and resource-dependent. Heart disease diagnosis is a multi-modal problem that requires integrating multi-source data such as clinical indicators, ECG, and imaging. However, integration faces challenges like heterogeneous formats and time alignment. Traditional prediction is mostly binary (diseased/healthy), while clinical practice requires more refined five-level grading, placing higher demands on the model's discriminative ability.

Clinical data (structured, low-dimensional, reflecting long-term state) and ECG signals (temporal, high-dimensional, reflecting real-time electrophysiology) are complementary. Feature engineering strategies: clinical feature standardization, non-linear transformation, and interaction feature construction; ECG feature extraction includes time-domain (R-R interval, QRS width, etc.), frequency-domain (power spectral density, etc.), and morphological features. Fusion strategies include early fusion (feature concatenation), late fusion (fusion after independent prediction of each modality), and middle fusion (hidden layer interaction).

Reasons for choosing random forest: it can handle high-dimensional features, perform non-linear modeling, has strong robustness, good interpretability, is not prone to overfitting, and naturally supports multi-classification (via voting or average probability decision). Model optimization: hyperparameter tuning (number of trees, maximum depth, etc.), class imbalance handling (class weights, SMOTE oversampling, stratified sampling), and stratified K-fold cross-validation to ensure stability.

Multi-class evaluation metrics: accuracy, macro-average F1, weighted-average F1, confusion matrix analysis (false negatives, false positives, etc.), and clinical-related metrics (sensitivity, specificity, AUC-ROC). Model interpretability: feature importance (Gini, permutation importance), individual prediction explanation (decision path, SHAP values, partial dependence plot). Clinical deployment is positioned as decision support, requiring the establishment of performance drift detection, data distribution monitoring, and feedback loop mechanisms.

Data preprocessing flow: cleaning → encoding → standardization → ECG preprocessing (filtering, denoising, R-peak detection) → feature extraction → feature selection → dataset partitioning. Model training code uses sklearn's RandomForestClassifier, combined with StratifiedKFold cross-validation and GridSearchCV for hyperparameter search. Feature importance visualization uses matplotlib and seaborn to display the Top20 features.

Current limitations: limited data scale, modality limitations (no imaging, etc.), generalization ability restricted across hospitals, and insufficient use of longitudinal temporal information. Future improvements: deep learning methods (CNN, RNN/LSTM, attention mechanism, multimodal Transformer), fusion of more data sources (ultrasound, wearables, genomics, electronic medical record text), and federated learning for multi-center data training under privacy protection.

Multimodal machine learning integrates clinical and ECG data, enabling a more comprehensive assessment of the disease compared to single modality. Random forest demonstrates good performance and interpretability.The project's value lies in its methodological significance: under limited resources (without large-scale imaging), reasonable fusion and feature engineering can build valuable medical AI systems. It provides practitioners with a complete process reference from data preparation to deployment, emphasizing the importance of reliability, interpretability, and ethical issues in medical scenarios.