Multi-Agent Systems Breakthrough in Screen Learning Behavior Analysis: A Comparative Study of Single-Agent vs. Multi-Agent Vision-Language Models

This article focuses on research using Vision-Language Models (VLMs) for automated analysis of screen learning behavior, comparing the performance of single-agent and multi-agent architectures in scene detection and action recognition tasks, proposing two innovative multi-agent frameworks and verifying their superiority, providing an efficient and scalable multimodal data analysis solution for the field of educational technology.

With the popularization of digital learning, screen learning behaviors (information retrieval, resource usage, knowledge creation) reflect cognitive and collaborative patterns, but traditional manual coding is time-consuming and inefficient. VLMs can process both visual and textual information simultaneously, bringing opportunities for automated analysis, but how to effectively apply them to complex learning behavior analysis remains a major challenge in academia.

The research designs solutions based on the ICAP framework (Passive/Active/Constructive/Interactive learning), which provides theoretical support for classifying learning behaviors. Multi-agent systems decompose tasks, allowing different agents to focus on specific domains, enhancing scene understanding and fine-grained action detection capabilities.

The experiment uses Claude-3.7-Sonnet, GPT-4.1 (closed-source), and Qwen2.5-VL-72B (open-source) models, comparing three types of architectures:

1. Single-agent: Directly processes complete recordings, facing challenges of context length limitations and high task complexity;
2. Workflow-based multi-agent: Collaboration among three agents (scene segmentation → behavior detection → verification);
3. Autonomous decision-making multi-agent: Inspired by ReAct, with iterative reasoning + tool calling + self-correction.

Workflow-based MAS: Sliding window for scene segmentation → behavior detection combined with cursor trajectory → verification of output consistency; task decoupling improves scene detection performance; Autonomous decision-making MAS: Maintains internal state, agents independently decide actions (analysis/segmentation/verification); ReAct paradigm enhances action recognition performance.

Multi-agent architectures outperform single-agent ones: workflow-based performs best in scene detection, autonomous decision-making is optimal in action recognition; the open-source Qwen2.5-VL-72B can compete with closed-source models under multi-agent configuration, reducing system costs.

Practical Significance: Online education can monitor learning engagement in real time and optimize collaborative grouping; researchers gain efficient video data analysis tools. Future Directions: Expand to programming/design collaboration scenarios, explore more agent collaboration modes, and reduce computing costs.

In complex multimodal tasks, architecture design is as important as model selection. A well-designed multi-agent system can outperform powerful single-agent VLMs, providing a reference for AI application development: prioritize architecture optimization rather than just pursuing large models.