Ensemble Learning for Diabetes Risk Prediction: A Practical Fusion of Decision Trees, Random Forests, and XGBoost

This project was published on GitHub by amirnazmi-gif on June 8, 2026 (link: https://github.com/amirnazmi-gif/Ensemble-Machine-Learning). Its core is to build an ensemble model by combining three machine learning algorithms—decision trees, random forests, and XGBoost—to improve the accuracy and robustness of diabetes risk prediction, providing data-driven intelligent decision support for early diabetes screening.

Diabetes is a common chronic disease worldwide. WHO data shows that there are over 400 million patients globally, and the incidence rate is rising. Early identification of high-risk groups is crucial for preventive intervention, but traditional assessments rely on experience, which are highly subjective and have limited accuracy. Machine learning technology can build prediction models by analyzing patient data to achieve precise risk stratification. This project adopts ensemble learning methods based on this need.

The project uses the PIMA Indians Diabetes Dataset (768 patient records). Features include physiological indicators (number of pregnancies, blood glucose concentration, blood pressure, etc.), biochemical indicators (family genetic tendency), demographic data (age), and the target variable (whether the patient has diabetes). Preprocessing steps: Missing values are filled with median or regression imputation; outliers are identified via box plots; features are standardized; class distribution is balanced if necessary.

Three complementary algorithms are selected as base models: 1. Decision trees (strong interpretability, captures non-linear relationships); 2. Random forests (Bagging strategy reduces overfitting, feature importance evaluation); 3. XGBoost (gradient boosting corrects errors, regularization controls complexity). Ensemble strategies include soft voting (weighted average of probability predictions), weight optimization (optimal weights determined via cross-validation), and optional Stacking strategy.

Evaluation metrics include accuracy, precision, recall, F1 score, AUC-ROC, and confusion matrix. Performance comparison: Decision trees serve as the baseline but are prone to overfitting; random forests have high stability; XGBoost as a single model has excellent performance; the ensemble model has comprehensive advantages, balancing accuracy and robustness. K-fold cross-validation is used to ensure evaluation reliability.

1. Early screening support: Integrate into physical examination systems to automatically assess risks and prioritize examinations for high-risk groups; 2. Personalized intervention: Develop plans for different risk levels (intensified intervention for high risk, diet and exercise guidance for medium risk); 3. Resource optimization: Prioritize resource allocation to high-risk patients to improve intervention cost-effectiveness.

Current limitations: Small dataset size, limited population representativeness, lack of features such as lifestyle, and no longitudinal prediction capability. Future directions: Integrate multi-center large-sample data, enrich features (lifestyle/genetics), explore deep learning, build time-series models, and adopt federated learning for privacy-preserving collaborative modeling.