VehicleTrack: An Intelligent Urban Traffic Analysis System from Offline Video to Trajectories

VehicleTrack is an end-to-end offline video traffic analysis pipeline that converts surveillance footage into vehicle trajectory data via computer vision. It addresses the challenges of traditional traffic data collection (high cost of inductive loops, limited coverage of floating cars, time-consuming manual observation) and provides quantitative metrics for urban policy-making. As a final project for the Urban Computing and Artificial Intelligence course at New York University, it demonstrates a low-cost and efficient approach to traffic analysis.

Traffic flow analysis is the foundation of policy-making in modern urban planning, but traditional collection methods have many issues: high installation and maintenance costs of inductive loops, limited coverage of floating car data, and time-consuming manual observation. With the popularization of urban surveillance cameras, video data has become an underutilized resource, but how to efficiently and accurately convert it into structured trajectory data is a challenge. The VehicleTrack project was born to address this pain point.

VehicleTrack adopts a modular end-to-end design, with key stages including:

1. Video preprocessing: Adaptive image enhancement and standardization to address issues of image quality, lighting, and perspective differences, laying the foundation for subsequent analysis;
2. Object detection and tracking: An optimized multi-object tracking algorithm that uses deep learning models to understand the semantic features of vehicles, handle scenarios such as occlusion and sudden lighting changes, and maintain tracking continuity.

The core technical challenge is converting pixel coordinates to real-world coordinates: Through camera calibration and inverse perspective transformation, the problem of perspective distortion from surveillance cameras is solved, enabling trajectory data to have real spatial scales and allowing accurate calculation of metrics such as driving speed, turning radius, and lane change frequency. Additionally, multi-camera trajectory stitching is achieved via spatiotemporal correlation algorithms to form complete travel records.

VehicleTrack converts trajectories into metrics understandable to decision-makers, including:

• Traffic volume statistics: Hourly and vehicle-type-specific traffic volume, identifying peak and off-peak periods;
• Speed distribution: Heat maps of road segment speeds, detecting speeding areas or congestion nodes;
• Turn ratio: Proportion of traffic flow in each direction at intersections, evaluating the rationality of lane configurations;
• Queue length: Vehicle accumulation during signal cycles, quantifying traffic capacity;
• Anomaly detection: Automatic identification of events such as wrong-way driving and long-term parking. These metrics can be imported into GIS, simulation software, or visualization platforms to support decision-making scenarios from micro to macro levels.

Advantages of offline processing:

1. Allows the use of more complex algorithms to achieve higher accuracy;
2. Fits the actual deployment environment of local storage for surveillance in many cities;
3. Reduces continuous occupation of network bandwidth and computing resources. Limitations: It cannot provide real-time early warning functions and needs to be paired with other real-time systems for immediate response scenarios such as traffic accident detection or violation capture.

As an academic project, VehicleTrack demonstrates the application of cutting-edge computer vision technology to urban problems. Its code structure is clear and documentation is complete, providing a reference for researchers. It proves that using existing surveillance resources and open-source tools, a practical traffic analysis system can be built, offering a low-cost and efficient data acquisition approach for urban planning departments, traffic consulting firms, and academic institutions, and contributing to the construction of smart cities.