Reliability-Aware Machine Learning for Physiological Signals: Maintaining Robust Predictions Amid Noise and Distribution Shifts

This article introduces a machine learning research framework for physiological signals (such as electrocardiograms, ECG), focusing on methods for evaluating and improving model reliability under conditions of noise, data corruption, and distribution shifts. The project takes ECG classification as an entry point, aiming to solve interference issues in real-world physiological signal data, ensure the reliability of model predictions, and is relevant to patient safety in medical applications.

In the healthcare field, machine learning models are widely used in physiological signal analysis such as ECG classification and heart rate monitoring. However, real-world data has issues like sensor noise, signal truncation, motion artifacts, and device differences. Traditional models perform well on clean data but their reliability drops significantly when encountering interference. The core problem is how to make models maintain reliable predictions under noise, corruption, and distribution shifts—this is not only a technical issue but also related to patient safety (e.g., an overconfident ECG classification model may mislead clinical decisions).

The project is a research-oriented scaffolding project, taking ECG classification as an entry point, using the MIT-BIH Arrhythmia Dataset, and designed with scalability in mind. The long-term goal extends to the analysis of biomarkers such as voice, cough, respiratory audio, and language signals. The vision is to develop reliable ML methods that can extract health information from noisy signals, with application scenarios ranging from physiological monitoring to African public health monitoring (aligned with the direction of Africa CDC, potentially used for tuberculosis monitoring, etc.).

The project establishes a multi-dimensional reliability evaluation system:

• Calibration metrics: Expected Calibration Error (ECE), calibration slope (measures the match between confidence and actual accuracy)
• Robustness metrics: Attenuation rate of metrics such as accuracy and precision after adding noise/corruption to test data
• Uncertainty metrics: Prediction entropy, confidence bounds (identifies samples requiring manual review)
• Brier score: A metric that combines accuracy and calibration

Robustness testing generates perturbations (Gaussian noise, signal missing, time masking, amplitude scaling) through a signal corruption pipeline, plots robustness curves of model performance attenuation, and identifies weak points.

The experimental workflow supports reproducibility:

• Synthetic data mode: Run the complete workflow without needing Kaggle datasets, quickly validate code
• Makefile commands: Encapsulate operations such as unit testing, synthetic experiments, and chart generation
• Configuration-driven: Experimental parameters are managed via configuration files, facilitating hyperparameter search
• Result visualization: Generate charts such as class distribution, waveform diagrams, corruption examples, and robustness curves

The technical implementation uses a modular structure: src/ (reusable modules), scripts/ (command-line scripts), configs/ (configuration files), notebooks/ (EDA guides), tests/ (testing), docs/ (documentation), suitable for research iteration and team collaboration.

Ethical Statement: The repository is for research and educational purposes only, not a clinical diagnostic system/medical device, and cannot replace medical judgment—this reflects respect for patient safety and ethical boundaries.

Future Directions: Expand from ECG to voice and cough audio biomarkers (e.g., COVID-19 screening, respiratory disease monitoring), as these signals also face challenges like noise. The project aligns with the research direction of Professor Visar Berisha's team at Arizona State University (in the field of reliable inference from noisy health signals).

This project demonstrates research-oriented AI engineering practice, focusing not only on prediction accuracy but also on reliability under complex real-world conditions. Through a systematic evaluation framework, reproducible experimental workflow, and emphasis on ethical boundaries, it provides a reference example for reliable machine learning research in the medical AI field. It is suitable for readers interested in medical AI, signal processing, model calibration, and robustness to explore in depth.