LuCID: A Data-Centric AI System for Predicting Cancer Risk in Diabetic Patients

LuCID is a longitudinal research project aimed at predicting the cancer risk of diabetic patients within the next three years using data-centric AI methods. This article analyzes the system's core design concepts, data processing workflow, model construction strategies, and multi-time window prediction mechanism, providing references for the application of medical AI in the field of chronic disease complication risk assessment.

The link between diabetes and cancer is a hot topic in medical research. Clinical data shows that the risk of certain cancers in diabetic patients is significantly higher than in the general population. Traditional risk assessment relies on single-time-point indicators, which makes it difficult to capture dynamic changes in the disease. The LuCID project uses data-centric AI methods to predict cancer risk by analyzing longitudinal laboratory data, providing a scientific basis for early intervention.

LuCID's data processing workflow includes:

1. Data Sources and Features: Covers demographic features (age, gender, BMI, etc.), longitudinal laboratory indicators (time-series data with timestamps such as HbA1c, HB, etc.), and outcome variables (cancer diagnosis labels, etc.);
2. Prediction Window Design: Calculate the corresponding age and feature values for the 0/1/2/3-year windows;
3. Summary Statistical Features: Compute mean, median, and standard deviation for each indicator (requires at least 5 test records);
4. Cancer Type Screening: Focus on the top 10 most common cancer types in the dataset to ensure sample size and clinical relevance.

LuCID's model construction strategies include:

1. Five-Fold Cross-Validation: Stratified data partitioning to ensure robustness;
2. Multi-Model Comparison: Test five models including Random Forest, XGBoost, LightGBM, Logistic Regression, and Linear SVM;
3. Class Imbalance Handling: Set class-weight parameters to focus on minority classes;
4. Threshold Optimization: Find the optimal threshold that balances sensitivity and specificity via ROC curves;
5. Multi-Window Fusion: Build independent models for four time windows and take the average of prediction probabilities as the final risk.

LuCID evaluates model performance using metrics such as ROC curves and AUC values, and provides a visual dashboard. Its clinical value is reflected in:

• Early Warning: Identify high-risk patients to support early screening;
• Personalized Medicine: Provide precise risk assessment based on longitudinal trajectories;
• Resource Optimization: Prioritize screening resources for high-risk groups to improve early detection rates.

The technical highlights of LuCID include well-designed feature engineering, sample screening strategies, multi-time window modeling, systematic model comparison, and class imbalance handling. This project is a successful application of data-centric AI in healthcare. Its methodology is not only applicable to cancer prediction but also provides a reusable framework for risk assessment of other chronic disease complications, demonstrating the potential of machine learning to transform into clinical decision-making tools.