PersonaDrive: Retrieval-Augmented Human-Style Autonomous Driving Agents for Diverse Traffic Behaviors in Closed-Loop Simulation

PersonaDrive: Retrieval-Augmented Human-Style Autonomous Driving Agents

Title: PersonaDrive: Retrieval-Augmented Human-Style Autonomous Driving Agents for Diverse Traffic Behaviors in Closed-Loop Simulation Abstract: This article introduces the PersonaDrive system, which uses retrieval-augmented generation technology to enable VLA autonomous driving agents to learn real human driving behaviors under different style instructions, achieving diverse traffic simulation where driving styles can be switched without retraining. Keywords: Autonomous driving, VLA, retrieval augmentation, driving simulation, behavior style, closed-loop simulation, CARLA, traffic agent Source Information:

• Original Author/Maintainer: arXiv authors
• Source Platform: arXiv
• Original Title: PersonaDrive: Human-Style Retrieval-Augmented VLA Agents for Closed-Loop Driving Simulation
• Original Link: http://arxiv.org/abs/2606.12616v1
• Publication Time: 2026-06-10T19:16:31Z

Background: Autonomous Driving Simulation Needs "Human-Like" Traffic Agents

The development and testing of autonomous driving systems rely on closed-loop driving simulators, where the behavioral authenticity of background traffic agents directly affects the credibility of the simulation. Background agents in existing simulators have single behaviors (rule-prescribed or trained in a single mode), which are far from real-world drivers with diverse styles (aggressive, conservative, etc.). How to endow agents with "human-like" qualities is a long-standing challenge.

Limitations of Existing Methods

Existing methods for introducing style variations have obvious limitations:

1. Post-hoc Annotation Method: Adding style labels to observed data after the fact only infers intent and cannot confirm the real style.
2. LLM Inference Reward Method: Using LLM to infer reward weights corresponding to styles is a proxy signal rather than real behavior demonstration. Common problem of both: Lack of real data of humans driving under explicit style instructions; style is an abstract inferred attribute rather than an actual learnable pattern.

Core Innovations

PersonaDrive is the first to use real data of humans driving under explicit style instructions (aggressive, neutral, conservative) to train VLA agents. Style here is an actual behavioral pattern rather than an inferred label, making it more realistic and controllable.

Technical Architecture: Three-Stage Training Process

1. Offline Triplet Mining: For each style of data, mine (query image, positive sample, negative sample) and use joint image-text similarity scoring to capture style visual cues.
2. Lightweight Retrieval Head Training: Freeze the pre-trained visual encoder, encode vehicle signals with a small control encoder, perform cross-modal fusion to form retrieval representations. The architecture is shared across styles, but each style has a different database.
3. VLA Backbone Fine-Tuning: Fine-tune a single VLA backbone. During inference, use retrieved context examples as behavior demonstrations to guide waypoint prediction.

Advantages of Style Switching During Inference

All styles share the same VLA backbone; switching styles only requires changing the database queried by the retrieval head:

• No retraining needed; just switch the database;
• Styles can be dynamically changed during simulation;
• Adding new styles only requires collecting corresponding human data and building a new database.

Experimental Results (Bench2Drive Benchmark)

1. Overall Performance Improvement: When no style is specified, driving score increases by 4.6% compared to SimLingo and 2.5% compared to HiP-AD.
2. Style-Specific Performance: Each style achieves the highest driving score, with only about 2% performance difference between styles; the strongest style performance of the best baseline DMW is 5.4% lower than the weakest style performance of PersonaDrive.
3. Behavioral Indicator Verification: From conservative to aggressive styles, average speed increases by 18% and acceleration increases by 25%, which aligns with human intuition.

Technical Contributions and Significance

• Data Aspect: First demonstrates the value of human driving data under explicit style instructions, which captures the essence of behavior more directly than post-hoc annotation or reward engineering.
• Architecture Aspect: The retrieval-augmented VLA architecture provides a new paradigm, combining retrieval and generation to achieve fine-grained style control.
• Application Aspect: Provides autonomous driving developers with a method to generate diverse real traffic scenarios, helping to test system robustness.

Conclusion

PersonaDrive makes simulation closer to the diversity of the real world, demonstrates the continuous value of human data in AI systems, and helps build safer and more reliable autonomous driving systems.

Limitations

1. Only three basic styles (aggressive, neutral, conservative) are tested; real styles are more complex;
2. Style switching is based on the entire database, with coarse granularity;
3. Experiments are mainly conducted in the CARLA simulation environment; the effectiveness of real-world data needs to be verified.

Future Directions

• Expand more style categories;
• Explore smooth transitions between styles;
• Apply to real-world driving data.