Zing Forum

Reading

Multimodal Heart Failure Prediction System Based on Graph Neural Networks

This article introduces an innovative framework for heart failure prediction that integrates ECG images and clinical data using graph neural networks, and details its technical architecture, multimodal fusion strategy, and clinical application value.

图神经网络心力衰竭预测多模态学习心电图分析医疗AIPyTorch GeometricResNet深度学习
Published 2026-06-05 00:14Recent activity 2026-06-05 00:20Estimated read 6 min
Multimodal Heart Failure Prediction System Based on Graph Neural Networks
1

Section 01

Introduction / Main Floor: Multimodal Heart Failure Prediction System Based on Graph Neural Networks

This article introduces an innovative framework for heart failure prediction that integrates ECG images and clinical data using graph neural networks, and details its technical architecture, multimodal fusion strategy, and clinical application value.

2

Section 02

Original Author and Source


3

Section 03

Research Background and Problem Definition

Heart Failure is one of the leading causes of hospitalization and death worldwide. Traditional diagnostic methods mainly rely on doctors' empirical judgment and single-modal examination data, making it difficult to fully utilize patients' multi-dimensional health information. With the development of deep learning technology, how to effectively integrate ECG image data and clinical numerical indicators to build more accurate prediction models has become an important research direction in the field of medical artificial intelligence.

The core problem proposed in this study is: How to introduce the non-Euclidean structure of patient relationships into the deep learning framework, and capture the similarity features between patients through Graph Neural Networks (GNN) to improve the accuracy of heart failure prediction?


4

Section 04

Technical Architecture and Overall Workflow

This project builds an end-to-end multimodal graph deep learning pipeline, and the overall architecture includes seven key stages:

5

Section 05

1. Data Cleaning and Validation

The system first filters damaged files and ensures that input images meet the minimum size requirement (100×100 pixels), laying a high-quality data foundation for subsequent processing.

6

Section 06

2. Class Balance and Data Augmentation

To address the class imbalance problem in the dataset (normal samples vs. patient samples), the ImageDataGenerator framework is used to expand the minority class. Through operations such as rotation, translation, scaling, and horizontal flipping, the number of samples in both classes is balanced.

7

Section 07

3. Clinical Feature Engineering

Five key medical variables are generated by sampling from pathological statistical distributions, which can reflect the patient's cardiac function status:

  • Ejection Fraction (EF): Normal distribution, mean 0.60 for healthy group, mean 0.38 for pathological group
  • Brain Natriuretic Peptide (BNP): Log-normal distribution, mean 4.2 for healthy group, mean5.8 for pathological group
  • NYHA Cardiac Function Classification: Categorical variable (levels 1-4), weighted by the severity of heart failure
  • Systolic Blood Pressure (SBP): Normal distribution, mean125 mmHg for healthy group, mean110 mmHg for pathological group
  • Age: Continuous integer sampling, mean55 years for healthy group, mean68 years for pathological group
8

Section 08

4. Multimodal Feature Fusion

This is the core technical link of the entire system. For visual feature extraction, a pre-trained ResNet18 backbone network (with the final fully connected classification layer removed) is used to generate a 512-dimensional embedding vector for each ECG image. The clinical feature vector (5-dimensional) is standardized using StandardScaler and then directly concatenated with the visual features to form a 517-dimensional comprehensive node feature vector.