IEEE Top Journal Review: Comprehensive Applications of Large Language Models in Autonomous Driving Scenario Testing

This systematic review published in the IEEE Transactions on Intelligent Transportation Systems comprehensively sorts out the applications of large language models (LLMs) in the entire process of scenario-based testing for autonomous driving systems, covering key links such as data augmentation, scenario generation, test execution, and safety assessment. The study fills the gap of lacking systematic reviews in this field and provides a new technical path for autonomous driving testing.

Safety verification of autonomous driving systems (ADS) is a core bottleneck for large-scale commercialization. Traditional road testing has high costs and long cycles, making it difficult to cover extreme scenarios; scenario-based testing (SBT) is a recognized verification method, but challenges still exist in scenario design and generation. LLMs, with their natural language understanding, code generation, and reasoning capabilities, provide a new path to solve these problems.

The study establishes five core stages of autonomous driving scenario testing:

1. Scenario Sources: LLMs are used for data augmentation, hazard analysis, automated annotation, and retrieval;
2. Scenario Generation: LLMs can convert natural language requirements into structured scenarios, extract scenario elements, generate standard formats (e.g., OpenSCENARIO), and executable scenarios;
3. Scenario Selection: Assist in clustering, sampling, and key scenario identification;
4. Test Execution: Undertake anomaly detection, environment configuration, parameter optimization, etc.;
5. ADS Evaluation: Participate in safety performance evaluation and generate structured reports.

• LLMs significantly improve the efficiency and diversity of scenario generation, outperforming traditional methods;
• Multimodal fusion (MLLMs) is an important direction, which can extract scenarios from videos, images, etc.;
• The combination of LLM generation and formal verification needs to be explored to improve test credibility.

Practical Value: Maintaining an open literature database (GitHub), establishing a unified terminology system, and covering cases from academia to industry. Industry Impact: Providing theoretical support for the evolution of testing standards, being recognized by IEEE top journals, and expected to become part of the industry-standard toolchain.

Limitations: Lack of large-scale industrial deployment data, immature physical consistency verification of scenarios, and need for standardization of test result reproducibility. Future Directions: Develop domain-specific LLMs, establish automatic verification pipelines, and explore collaboration with technologies such as digital twins.

Large language models are reshaping the paradigm of autonomous driving testing. This review provides a comprehensive technical map, and the deep integration of the two will spawn more intelligent and efficient verification methods, accelerating the implementation of autonomous driving technology.