DynamicVL: A Multimodal Large Language Model Evaluation Benchmark for Dynamic Urban Environments

The DynamicVL project establishes a benchmark specifically for evaluating the ability of multimodal large language models (MLLMs) to understand dynamic urban environments, promoting the development of urban data analysis technologies.

Cities are dynamic complex systems, and understanding urban environments is crucial for applications such as autonomous driving, urban planning, and intelligent transportation. However, existing MLLM benchmarks mostly focus on static scenarios and lack specialized evaluation for dynamic urban environments.

DynamicVL is a benchmark specifically designed to evaluate the ability of multimodal large language models to understand dynamic urban environments:

• Temporal Understanding: Changes in urban environments over time
• Dynamic Object Tracking: Moving pedestrians, vehicles, etc.
• Scene Semantic Understanding: Identification of urban functional areas
• Event Reasoning: Understanding of urban activities and events

• Autonomous driving system evaluation
• Urban surveillance video analysis
• Smart city application development

Dynamic urban environments pose unique challenges:

1. Lighting Changes: Impact of day/night cycles and weather
2. Occlusion Issues: Blockages by buildings and vehicles
3. Complex Interactions: Dynamic interactions among multiple entities
4. Long Temporal Dependencies: Temporal correlations of events

DynamicVL fills the gap in MLLM evaluation and provides a standardized assessment tool for developing more robust urban perception AI systems.

• GitHub Repository: https://github.com/anggaumhar/dynamicvl