V-CAST: Curvature-Aware Spatio-Temporal Pruning Method for Efficient Video Large Language Models

V-CAST proposes a training-free, plug-and-play Token pruning strategy for video large language models. Through a curvature-guided temporal allocation mechanism and a dual-anchor spatial selection mechanism, it maintains 98.6% of the original performance while reducing peak memory and total latency to 86.7% and 86.4% of the Qwen3-VL-8B-Instruct baseline, respectively, solving the Token explosion problem.

Efficiency Challenges

VideoLLMs perform strongly across multiple scenarios, but the massive volume of video data leads to Token explosion, resulting in a large context during the pre-filling phase and a sharp increase in computation and memory overhead.

Dilemmas in Token Compression

Limitations of existing methods: Coarse-grained frame-by-frame allocation ignores content dynamics; scene segmentation easily causes information fragmentation; Token merging leads to MRoPE coordinate misalignment, affecting spatio-temporal reasoning.

Curvature-Guided Temporal Allocation

Model Token compression as trajectory approximation, using curvature to reflect content changes: identify high-curvature semantic turning points, perceive event boundaries, and dynamically allocate Token budgets (fewer allocations for smooth segments, more for intense segments).

Dual-Anchor Spatial Selection

Preserve high-entropy visual regions without interfering with attention, maintain the original spatio-temporal coordinates of Tokens, and avoid coordinate misalignment.

Accuracy Preservation

Achieves 98.6% of the original performance across multiple tasks, with an average improvement of 1.1% over the second-best method.

Efficiency Improvement

Peak memory is reduced to 86.7% of the baseline, and total latency to 86.4%.

Cross-Architecture Compatibility

Training-free and plug-and-play, applicable to VideoLLMs of different architectures and scales.

Facilitates:

• Real-time video analysis (low latency supports real-time responses);
• Edge device deployment (reduces memory usage);
• Long video processing (avoids Token explosion);
• Cloud cost optimization (improves efficiency and reduces costs).

Limitations

Curvature calculation incurs additional preprocessing overhead.

Future Directions

Optimize curvature calculation, explore synergy with fine-tuning, integrate audio cues, and support dynamic resolution input.

V-CAST balances accuracy and efficiency through innovative mechanisms, and its training-free, plug-and-play feature allows for rapid application, paving the way for the practical deployment and scaling of video understanding technologies.