US Flight Delay Prediction System Based on 2.9 Million Flight Data: From Data Cleaning to Interactive Visualization

This article introduces an open-source project that integrates 2.9 million U.S. domestic flight records and builds a complete pipeline for data processing, interactive visualization dashboard, and machine learning prediction. The project can predict flight delays and arrival times, and has practical value for passenger travel, airline operation optimization, etc. The original author of the project is Hessam Asadi, sourced from GitHub, with the original title US-Flight-Delay-Dashboard-Predictor.

The aviation industry suffers billions of dollars in economic losses each year due to flight delays, and accurate delay prediction is a core issue for operational optimization. The project's data source is from Kaggle, with the original data containing about 3 million U.S. domestic flight records from 2019 to 2023. After cleaning (removing outliers, canceled flights, and invalid routes), 2.87 million high-quality records are retained, covering 18 major airlines and 340 U.S. domestic airports.

The project uses Streamlit to build an interactive dashboard, which includes three major modules:

1. Airport Distribution Map: Built with Folium, using color coding for average takeoff delays (green for on-time, red for delayed), marker size proportional to flight volume, and supporting heatmap mode;
2. Airline Analysis: Displays the worst/best airport rankings for the selected airline, bar charts of average delays, and summary statistics;
3. Airport Comparison: Horizontal comparison of up to 10 airports, including indicators such as total flights, average delay, and on-time rate, with support for CSV export.

The core of the project is a random forest prediction model. Input features include departure/destination airport, airline, day of the week, departure hour, and month. Feature importance analysis shows: departure hour (36%) > departure airport (23%) > destination airport (15%) > airline (14%) > day of the week and month. Model performance: regression MAE of 14 minutes, classification accuracy of 67.3%, recall rate of delayed flights of 64.4%, which can predict the number of delay minutes and the probability of delay exceeding 15 minutes, and estimate arrival time.

The technology selection balances efficiency and performance:

• Data layer: Pandas, NumPy (cleaning and preprocessing);
• Visualization layer: Folium (geographic visualization), Plotly (interactive charts);
• Web application: Streamlit (dashboard framework);
• Machine learning: Scikit-learn's RandomForestRegressor/Classifier;
• Class balance: Adopted class balance strategy for the imbalance between delayed/on-time samples.

Practical Value:

• Passengers: Evaluate the delay risk of routes and time slots before booking tickets;
• Airlines: Identify operational bottlenecks and optimize scheduling;
• Airports: Analyze the gap between themselves and industry benchmarks;
• Researchers: Reference for complete data processing and modeling. Future Expansion: Introduce real-time weather data, add features such as previous flight status, try deep learning models, and build API services for third-party calls.

The US-Flight-Delay-Dashboard-Predictor project demonstrates the complete process of converting massive historical data into actionable insights and predictive capabilities, covering data cleaning, feature engineering, visualization exploration, machine learning modeling, and other links. It embodies the methodology of data science projects and is an excellent reference case for getting started with aviation data analysis or learning end-to-end data projects.