Machine Learning Practice for Industrial Predictive Maintenance: From Data Exploration to Model Optimization

This project focuses on machine learning practices for industrial predictive maintenance, using sensor telemetry data from the AI4I 2020 dataset. It compares models like Logistic Regression, Decision Tree, Random Forest, and XGBoost to address core challenges including imbalanced classification (extremely few failure samples), trade-off between recall and precision, model interpretability, and business implementation. The goal is to shift from reactive maintenance to proactive prevention and reduce operational costs.

Unexpected failures of industrial equipment lead to huge costs such as production downtime, maintenance delays, and safety risks. Traditional post-failure maintenance strategies cannot prevent these issues. Predictive maintenance identifies failures in advance through telemetry data, but there are key trade-offs:

• False Negatives (Missed Detection): Cause losses from unexpected downtime;
• False Positives (False Alerts): Trigger unnecessary maintenance and waste resources. Industrial datasets generally have severe imbalance (extremely low proportion of failure samples), making the accuracy metric ineffective. Thus, we need to focus on recall, precision, and F1 score.

Dataset: Uses the AI4I 2020 Predictive Maintenance Dataset, which includes features like air temperature, process temperature, rotational speed, torque, and tool wear. The target variable is machine failure (0 = no failure, 1 = failure). Model Selection: Compares four types of models:

1. Logistic Regression (linear baseline, strong interpretability);
2. Decision Tree (non-linear, intuitive and easy to understand);
3. Random Forest (ensemble learning, good robustness);
4. XGBoost (gradient boosting, excellent performance). Imbalance Handling: Uses strategies like oversampling (SMOTE), undersampling, class weights, and threshold adjustment to balance recall and precision.

Model Performance Comparison Table:

Model	Recall	Precision	F1 Score
Logistic Regression	0.80	0.13	0.23
Decision Tree	0.61	0.68	0.64
Random Forest	0.43	0.91	0.58
XGBoost (Recall Optimized)	0.95	0.23	0.37
XGBoost (F1 Optimized)	0.76	0.70	0.73
Key Conclusion: XGBoost optimized for F1 achieves the best balance between failure detection and false alert control, so it is finally selected.

Model Interpretability: Reveals model decision logic through SHAP values and feature importance analysis, helping engineers understand the basis of predictions. Error Analysis: Studies false negatives (missing failure patterns) and false positives (misjudging normal states) to identify directions for improving data quality or feature engineering. Business Trade-offs: Need to consider maintenance costs vs. downtime losses, industry tolerance for false results, and the need for regular model retraining. Limitations: There is significant feature overlap between failure and non-failure cases, and instantaneous telemetry data has limitations. Time series methods like LSTM/Transformer need to be introduced.

Feature Engineering:

• Add sliding window statistical features (mean, variance, trend);
• Introduce domain knowledge features (e.g., temperature-rotational speed interaction term);
• Standardize/normalize numerical features. Model Optimization:
• Try time series models to capture temporal dependencies;
• Integrate prediction results from multiple models;
• Implement online learning to adapt to data drift. Deployment Considerations:
• Build a performance monitoring dashboard;
• Set confidence thresholds and route low-confidence samples to manual review;
• Regularly collect feedback data for continuous model optimization.

This project demonstrates the complete ML workflow for industrial predictive maintenance: business understanding → data exploration → model selection → optimization → error analysis. Key takeaways:

1. F1 score is more meaningful than accuracy in imbalanced classification;
2. Model optimization needs to align with the business cost structure;
3. Interpretability is the foundation of trust for industrial deployment;
4. Feature overlap limits models using instantaneous data, so time series methods are worth exploring. It provides a complete reference framework from data science to business implementation for industrial AI practitioners.