SafeVL: A Fine-Grained Reasoning Framework for Driving Safety Assessment Based on Vision-Language Models

SafeVL is a driving safety assessment framework developed by SaFo-Lab, integrating object detection (Grounding DINO), segmentation (SAM2), and reasoning capabilities of vision-language models (Qwen series) to achieve fine-grained scene understanding and intelligent safety analysis. The project was released on GitHub on May 31, 2026, aiming to address the safety assessment challenges of complex road conditions in the development of autonomous driving technology.

With the rapid development of autonomous driving technology, accurately assessing the safety of driving scenarios has become a key challenge. Traditional rule-based methods struggle to handle complex and changing road conditions. In recent years, vision-language models (VLMs) have demonstrated strong visual understanding and reasoning capabilities, providing a new path for driving safety assessment. The SafeVL project combines the reasoning capabilities of VLMs with specialized object detection and tracking technologies to build a refined intelligent system for safety assessment.

SafeVL adopts a modular architecture that integrates visual perception and language reasoning:

Visual Perception Layer

Using Grounding DINO (open-vocabulary object detection model) combined with SAM2 (accurate segmentation and tracking), it achieves robust perception of dynamic driving environments. Its advantages include flexible text-guided detection, pixel-level segmentation accuracy, and continuous tracking capabilities.

Reasoning and Analysis Layer

Based on the Qwen series VLM for deep reasoning, it can identify dangerous scenarios and interaction relationships, evaluate safe distances and trajectories, and generate structured safety assessment reports.

SafeVL's main application scenarios include:

Driving Behavior Analysis

Identifying dangerous driving tendencies (such as following too closely, improper lane change timing, insufficient attention to vulnerable road users);

Scene Risk Assessment

Fine-grained assessment of complex scenarios (conflict points at intersections, prediction of pedestrian crossing intentions, visibility analysis in bad weather);

Autonomous Driving System Validation

Verifying the rationality of decisions, discovering edge cases, and providing data support for system improvements.

The SafeVL codebase includes key modules: api/ (external interfaces), blg/ (business logic), config/ (configuration management), dataset/ (data processing), safevl/ (core algorithms), src/ (source code). The development toolchain includes gradio_demo.py (interactive demo), quickstart.ipynb (quick start guide), inference.py (inference script), and test.py (test cases).

SafeVL represents an important development direction in the field of driving safety assessment. Compared with traditional rule-based or statistical methods, it achieves assessment closer to human cognition through VLM reasoning. Its advantages include:

1. Interpretability: The reasoning process is transparent, facilitating debugging and improvement;
2. Generalization ability: Open-vocabulary detection and general VLM adapt to new scenarios;
3. Modular design: Components can be optimized and replaced independently, which is conducive to iteration;
4. Practicality: Complete code and demo tools lower the threshold for research and application.

SafeVL demonstrates the great potential of VLMs in the field of driving safety. Combining advanced visual perception and deep reasoning, it provides a promising solution for autonomous driving safety assessment. In the future, with the progress of VLM technology, it is expected to assist driving decisions in real time and improve the overall safety of road traffic. For researchers and developers, SafeVL provides a reference implementation of cutting-edge AI technology applied to traffic safety issues.