FlashVID: A Training-Free Efficient Acceleration Scheme for Video Large Language Models

FlashVID is a training-free acceleration scheme for video large language models. Its core uses a tree-structured spatiotemporal token merging strategy to increase inference efficiency by several times without retraining the models, while maintaining high-quality output. This scheme is an ICLR 2026 Oral paper and has been open-sourced. It has advantages such as training independence and flexible deployment, and is applicable to various pre-trained video LLMs.

Video Large Language Models (Video LLMs) have made significant progress in recent years, but they face the challenge of huge inference computation overhead due to the massive spatiotemporal information in videos. Traditional acceleration methods require retraining the models, which are costly and may affect performance. The core insight of FlashVID is to optimize the token processing method during the inference phase, reducing computational complexity by merging redundant spatiotemporal tokens without sacrificing quality.

FlashVID's core innovation is the tree-structured spatiotemporal token merging strategy:

Spatial Dimension Merging

Identify visual redundancy in frames and merge similar visual tokens through tree clustering to reduce the number of tokens per frame.

Temporal Dimension Compression

Leverage the temporal continuity of videos to dynamically merge redundant tokens at different time scales—preserve details in dynamic regions and aggressively compress static regions.

Training-Independent Feature

Works entirely in the inference phase, requiring no additional training costs. It can be applied to any pre-trained video LLM without training bias, enabling flexible deployment.

Implementation Details

Includes a token importance evaluation module (based on attention mechanism) and a tree construction algorithm, using an adaptive threshold mechanism: adjust the merging aggressiveness according to video content and task type (e.g., conservative for action recognition, aggressive for high-level semantic understanding).

As an ICLR 2026 Oral paper, FlashVID has been rigorously experimentally validated: while maintaining output quality, the inference speed is increased by several times. The acceleration effect becomes more significant as video length and resolution increase. The computational savings for long videos and high-resolution inputs show non-linear growth, making it suitable for complex video tasks.

FlashVID's application scenarios include:

• Real-time video understanding systems: Improve user experience;
• Cloud video services: Significantly reduce costs;
• Mobile devices: Enable complex video AI functions.

Its training-independent feature makes it highly versatile, applicable to any Transformer-based video LLM, lowering the deployment threshold for advanced video AI.

FlashVID has been open-sourced on GitHub, providing core algorithm implementations, documentation, and examples to facilitate reproduction and extension. In the future, it can be combined with hardware optimization, quantization technology, etc., to further improve efficiency. As the proportion of video content grows, such efficient inference technologies will become increasingly important.