How Large Language Models Revolutionize Scenario-Based Testing for Autonomous Driving: A Deep Dive into the IEEE T-ITS Survey

This article interprets the survey paper titled LLM4ADSTest: A Survey on the Application of Large Language Models in Scenario-Based Testing of Automated Driving Systems published by the Graz University of Technology team in the IEEE Transactions on Intelligent Transportation Systems (T-ITS) (GitHub repository: https://github.com/ftgTUGraz/LLM4ADSTest, preprint: https://arxiv.org/pdf/2505.16587). The survey systematically outlines the applications of large language models (LLMs) throughout the entire workflow of scenario-based testing for autonomous driving, covering key stages such as scenario generation, data annotation, hazard prediction, and safety assessment, and discusses the current research status and future trends in the field.

The safety verification of autonomous driving systems (ADS) faces the dilemma of high cost in traditional road testing and difficulty in covering all hazardous scenarios (requiring hundreds of millions of kilometers of driving to statistically prove safety). Scenario-based testing verifies performance by constructing scenarios in simulations, but it relies heavily on manual work, is inefficient, and struggles to exhaust edge cases. The natural language understanding, code generation, and knowledge reasoning capabilities of LLMs bring new opportunities to solve these problems.

A scenario refers to a time-series description that includes elements such as the ego vehicle, traffic participants, road environment, weather, and lighting. It is divided into four levels: functional, abstract, logical, and concrete. The scenario testing lifecycle consists of five stages: scenario source acquisition, generation, screening, test execution, and system evaluation. The application potential of LLMs runs through all these stages.

LLMs have three main applications in the scenario source stage: 1. Hazard Analysis and Risk Assessment (HARA): Automatically identify potential hazardous scenarios from accident reports and regulatory documents; 2. Data Annotation: Generate semantic labels for sensor data (e.g., traffic participant intent, road topology); 3. Data Retrieval: Quickly locate simulation data through natural language queries.

Scenario generation is the core of LLM applications, with four roles: 1. Human-Computer Interaction Interface: Convert engineers' natural language intentions into structured scenarios; 2. Data Interpreter: Extract information from unstructured data such as accident reports to generate simulation scenarios; 3. Intermediate Format Generator: Convert scenario description standards (e.g., "OpenSCENARIO", CommonRoad); 4. Executable Scenario Generator: Generate simulation code via template filling, end-to-end code generation, or hybrid methods.

Test Execution Stage: LLMs monitor abnormal simulation behaviors in real time and predict potential collision risks; automatically generate simulation configuration files (maps, vehicle models, sensor parameters). Evaluation Stage: LLMs evaluate the rationality and compliance of ADS decisions based on human driving experience; test the system's ability to understand and reason about traffic rules through interactive Q&A to assess "driving intelligence".

The GitHub repository LLM4ADSTest contains dozens of related studies, categorized by application scenarios, with annotations of publication venues, dates, code links, and core contributions. The community supports nominating new papers via Google Forms, reporting errors through Issues, or contributing content via PRs to ensure the timeliness and comprehensiveness of the resources.

LLM applications face four major challenges: 1. Interpretability and Credibility (compliance with physical laws of scenarios); 2. Standardization and Interoperability (unified scenario description format); 3. Real-Time Performance and Efficiency (model lightweighting); 4. Safety and Ethics (avoiding simulation results misleading real-world decisions). LLMs are reshaping the paradigm of autonomous driving testing, addressing pain points of traditional methods such as high cost and low coverage. This survey provides a roadmap for researchers, and the open-source library builds a platform for community collaboration.