LLM Drift: A Study on Behavioral Drift Phenomena of Large Language Models in Adversarial Multi-Agent Interactions

This article focuses on the behavioral drift (LLM Drift) phenomenon of large language models in adversarial multi-agent interactions. It uses LangGraph to build an adversarial debate simulation platform and combines 22-dimensional behavioral metrics to evaluate drift across five dimensions—psychometric, personality traits, emotional state, cognitive structure, and social relations—providing systematic tools and methods for understanding AI behavioral changes.

With the widespread deployment of LLMs in various scenarios, the LLM Drift phenomenon—where model behavior deviates from initial settings in adversarial interactions—has gradually emerged. The traditional assumption is that model behavior is stable, but in practice, during long-term adversarial interactions, even with clear role instructions, models still exhibit measurable drift, affecting application scenarios that rely on role consistency such as virtual companionship and educational tutoring.

LLM Drift refers to the measurable changes in a model's behavior that deviate from its settings during dialogue in specific role tasks. It is studied across five dimensions:

1. Psychometric dimension: Focuses on logical stance (analytical, authoritative, etc.);
2. Personality traits dimension: Evaluates openness, conscientiousness, etc., based on the OCEAN model;
3. Emotional state dimension: Tracks emotional load (tone, arousal level, etc.);
4. Cognitive structure dimension: Examines vocabulary diversity, reasoning depth, etc.;
5. Social relations dimension: Analyzes power dynamics, empathy ability, etc.

The research uses a five-stage pipeline:

1. Research definition: Clarify behavioral vectors and evaluation framework;
2. Adversarial simulation: Build a two-team debate architecture based on LangGraph, including role, thinking, and critical agents;
3. Data archiving: Save complete dialogue and memory data;
4. Quantitative evaluation: Use the RAGAS framework and Gemini model to score 22-dimensional metrics;
5. Analysis and visualization: Present drift trajectories via a Streamlit dashboard. Technical highlights include state machine-driven debate flow, hierarchical scoring system, and multi-level memory architecture.

The core hypothesis (adversarial pressure leads to systematic drift) has important implications:

• For developers: Need to regularly calibrate role consistency, add additional stability mechanisms for adversarial scenarios, and pay attention to emergent behaviors of multi-agents;
• For AI safety researchers: Provides a framework for studying model behavior boundaries and failure modes.

The platform supports:

• Custom debate topics and role settings;
• Adding new drift evaluation dimensions;
• Replacing underlying models (e.g., different versions of Gemini);
• Adjusting simulation parameters (temperature, number of rounds, etc.). Its modular design is suitable for a wide range of dialogue behavior analysis.

This project combines psychological theory, multi-agent simulation, and quantitative evaluation to provide tools for understanding AI behavioral changes under pressure. As LLMs are deployed in critical scenarios, understanding and predicting behavioral drift will become an important part of AI system design and safety assessment.