eta-engine: A New York Taxi Trip Time Prediction System Fusing Neural Networks and LightGBM

In urban traffic management and ride-sharing services, accurately predicting taxi trip times is a key challenge. Traditional methods struggle to capture complex dynamics, and single models have limited performance. The eta-engine project proposes an innovative solution: fusing deep neural networks with LightGBM gradient boosting trees to achieve accurate prediction of New York taxi trip times by learning spatial embedding representations, combining the advantages of both model types to improve prediction performance.

As one of the world's largest taxi markets, New York City generates millions of trip records with spatiotemporal information daily. Trip time prediction faces multiple challenges: highly dynamic urban traffic (fluctuations caused by weather, events, etc.), complex spatial relationships (describable by non-Euclidean distances), and noise and outliers in data. To address these challenges, eta-engine builds a complete machine learning system that extracts spatial embeddings and combines the advantages of multiple algorithms for prediction.

eta-engine adopts an ensemble architecture of neural networks and LightGBM, with complementary advantages:

• Neural networks: Learn complex spatial relationships and nonlinear feature interactions; convert discrete geographic locations into continuous vectors via embedding layers to automatically capture latent relationships.
• LightGBM: Process structured features (time, distance, high-level features from neural networks) efficiently and with interpretability. The two models are fused at the feature level (embeddings as input to LightGBM), and prediction results can be weighted averaged or stacked to improve robustness.

Traditional methods use latitude/longitude or fixed partitions, lacking semantic information. eta-engine uses data-driven embedding: each geographic location is a learnable vector, optimized based on trip times during training. For example, the vectors of origin and destination for fast trips are close, while those for long-duration trips are far apart. Without manual rules, it automatically discovers patterns from historical data (e.g., clustering of commercial centers and transportation hubs) and captures rich semantics such as regional functions and traffic connections.

Feature Engineering: Decompose time features (hour, week, month); distance features (Haversine, Manhattan distance); integrate historical statistics (average time for specific time periods/routes). Data Preprocessing: Filter unreasonable trips (negative time, excessively long duration), records outside New York, duplicate or corrupted data. Training Strategy: Neural network pre-training (learn initial embeddings) → end-to-end fine-tuning; LightGBM uses cross-validation, hyperparameter search, and early stopping to prevent overfitting; ensemble strategies (simple average, weighted average, stacking) to improve accuracy.

The practical value of eta-engine is extensive:

• Ride-sharing platforms: Improve ETA prediction accuracy, optimize user experience and dispatch efficiency.
• Urban planning: Identify congestion hotspots and bottlenecks, assist in traffic signal timing and infrastructure planning.
• Driver decision-making: Support selecting favorable orders and planning optimal routes to maximize revenue.

Technical Insights: The fusion of deep learning and traditional machine learning (representation learning + structured data processing) can be extended to spatiotemporal problems such as delivery time prediction and bus arrival estimation; open-source implementation provides best practice references. Future Outlook: Expand multi-modal data (real-time traffic, weather); model road topology using GNNs; introduce reinforcement learning to implement dynamic prediction strategies, moving toward more intelligent and accurate traffic prediction.