GliDe: An Open-Ended Game Vulnerability Detection Framework Based on Agent Reasoning and Temporal Localization

This article proposes the GliDe framework (based on agent reasoning and temporal localization) and the VideoGlitchBench benchmark, which for the first time achieve open-ended detection, natural language description, and precise temporal localization of vulnerabilities in game videos, significantly improving the performance of multimodal models on game anomaly detection tasks. This achievement addresses the limitations of traditional detection methods and provides new directions for fields such as game testing automation.

Video game vulnerabilities disrupt user experience or economic balance. Traditional methods relying on manual testing or rule matching struggle to handle complex interactions and massive content. Existing AI methods are limited to image classification or closed-ended question answering; they cannot understand game mechanics, distinguish between vulnerabilities and normal anomalies, or precisely localize temporal intervals, making them hard to meet real-world scenario needs.

The research team built VideoGlitchBench, which contains 5238 video clips from 120 games, each annotated with vulnerability descriptions and time spans. The construction process is rigorous: collect multi-type game recordings → professionally annotate abnormal behaviors and descriptions → mark time points. Its "open-ended" design requires generating free text, which is closer to practical applications and tests the model's real understanding ability.

The GliDe framework is based on an agent architecture and includes three components:

1. Game-aware Context Memory: Dynamically stores knowledge such as game types and gameplay, combined with prior reasoning (e.g., distinguishing between wall-clipping vulnerabilities and skills);
2. Debating Reflector: Generates candidate explanations from multiple perspectives and conducts debates to identify subtle differences and improve conclusion reliability;
3. Event-level Temporal Localization: Aggregates key frames/state changes from bottom to top, outputting precise vulnerability time intervals and descriptions.

The evaluation protocol examines semantic fidelity (description completeness, accuracy, fluency) and temporal accuracy (start/end point deviation, overlap), ensuring that the model generates understandable descriptions and precise localizations to meet the practical needs of game testing.

Open-ended detection is extremely challenging for multimodal models, and baseline models' performance is far from practical. GliDe achieves significant improvements in detection accuracy, description quality, and temporal precision, verifying the value of the agent architecture. Current model weaknesses: poor cross-frame reasoning and easy misjudgment in understanding complex game mechanics, which point to directions for future research.

GliDe and the benchmark promote game testing automation (24/7 scanning, cost reduction/efficiency improvement), and can be extended to fields such as content moderation, anomaly monitoring, and experience optimization. In the future, AI-assisted quality management will become an industry standard.

VideoGlitchBench and GliDe lay the foundation for open-ended game vulnerability detection, demonstrating the potential of agent reasoning and temporal localization. With the advancement of multimodal models, AI will become a powerful assistant for developers, helping to create more stable and smooth game experiences.