VEBench: A Large Multimodal Model Evaluation Benchmark for Real-World Video Editing Scenarios

VEBench is the first benchmark to systematically evaluate the video editing understanding and operational reasoning capabilities of Large Multimodal Models (LMMs), containing 3.9K high-quality edited videos (with a total duration of over 257 hours) and 3,080 human-validated question-answer pairs. Experiments reveal a significant gap between current models and human-level editing cognition, pointing the way for the development of intelligent video editing systems.

Video editing integrates technology, art, and narrative, requiring multimodal reasoning capabilities (selecting materials, determining timeline positions, and combining into a coherent narrative). Existing LMMs have made progress in general video understanding (recognizing objects/actions, answering questions), but lack the "selection" and "combination" capabilities required for editing; existing benchmarks only focus on passive understanding and do not cover active creation needs.

Benchmark Design

VEBench includes two main tasks:

1. Technical Recognition: Test the model's ability to recognize and understand 7 core editing techniques (jump cut, match cut, etc.);
2. Operational Simulation: Require the model to select appropriate clips from candidate materials, locate timeline positions, and explain the reasons.

Dataset and Annotation

• 3.9K+ real-scenario videos (documentaries, short videos, etc.);
• 3080 human-validated question-answer pairs;
• Three-round annotation process: AI-assisted pre-annotation → expert manual review and correction → cross-validation and consistency check.

Technical Recognition Task

• The best model (Gemini-2.5-Pro) achieves an average accuracy of 65%, while human experts reach 92% (a 27-percentage-point gap);
• Easy-to-recognize techniques: jump cut, dissolve (obvious visual features); hard-to-recognize: match cut, L/J cut (require semantic or audio-visual association understanding).

Operational Simulation Task

• The best model has a selection accuracy of 45% and positioning accuracy of 38%, compared to human experts' 88% and 85% respectively.

Error Patterns

Time reasoning failure, lack of narrative coherence, intent understanding deviation, insufficient context utilization.

1. Gap from Perception to Creation: Requires goal-oriented reasoning, counterfactual thinking, and aesthetic judgment;
2. Multimodal Integration Complexity: Needs to integrate visual coherence, audio design, narrative rhythm, and emotional arc;
3. Long-term Temporal Reasoning: Requires considering past narrative accumulation, current effects, and future directions, testing the model's memory and planning capabilities.

1. Formalization of Editing Knowledge: Build structured knowledge bases, learn professional editing knowledge, and transform implicit knowledge;
2. Creative Reasoning Capability: Develop creative evaluation mechanisms, explore human-machine collaboration models, and integrate human aesthetic preferences;
3. Interactive Editing Assistant: Provide candidate clip suggestions, explain reasoning processes, and learn from feedback;
4. Multi-Agent Editing System: Different agents focus on subtasks (material selection, audio design, etc.), with humans coordinating the team.

VEBench reveals the gap between LMMs and humans in video editing capabilities, which is both a challenge and an opportunity:

• For researchers: Promote cutting-edge research in multimodal reasoning, creative AI, long-term temporal understanding, etc.;
• For industry: Lower the threshold of professional production, empower creators, and automate content production; VEBench, through high-quality evaluation data and benchmarks, helps advance intelligent video editing technology, with the goal of enabling AI to master the "transformative" editing power.