# HAWK: A New Breakthrough in Visual Token Pruning for Multimodal Large Models

> HAWK proposes a visual token pruning method based on the importance of attention heads. It achieves 80% visual token pruning without training while maintaining 96% of the original accuracy, providing a feasible solution for the real-time deployment of multimodal large models.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-04-09T05:09:22.000Z
- 最近活动: 2026-04-10T01:46:33.179Z
- 热度: 130.4
- 关键词: 多模态大模型, 视觉Token剪枝, 注意力机制, 模型压缩, 推理优化, Qwen2.5-VL, Transformer, 计算效率
- 页面链接: https://www.zingnex.cn/en/forum/thread/hawk-token
- Canonical: https://www.zingnex.cn/forum/thread/hawk-token
- Markdown 来源: floors_fallback

---

## HAWK: A New Breakthrough in Visual Token Pruning for Multimodal Large Models (Introduction)

HAWK proposes a visual token pruning method based on the importance of attention heads. It achieves 80% visual token pruning without training while maintaining 96% of the original accuracy, providing a feasible solution for the real-time deployment of multimodal large models.

## Background: Efficiency Dilemma of Multimodal Large Models

Multimodal Large Language Models (MLLMs) are powerful, but high-resolution visual inputs lead to an explosive growth in the number of visual tokens, significantly increasing inference latency, computing resource consumption, and GPU memory usage. This makes it difficult to deploy them in real-time scenarios such as autonomous driving, robot control, or instant visual question answering on mobile devices.

## Current Status: Limitations of Traditional Visual Token Pruning

Traditional visual token pruning assumes that all attention heads contribute equally, ignoring the functional differentiation of different attention heads in the Transformer architecture—some focus on shape contours, some on color textures, and others on spatial positional relationships. This leads to failure to consider the differentiated contributions of heads when evaluating token importance.

## Core Innovation of HAWK: Head Importance-Aware Pruning

HAWK evaluates the importance of visual tokens through two dimensions: 1. Head importance weight: dynamically calculates the activation intensity and stability of each attention head to assign weights; 2. Text-guided attention: identifies relevant tokens in combination with the current text task. Moreover, it requires no training and can be directly applied to pre-trained MLLMs, making it plug-and-play.

## Experimental Evidence: Dual Improvement in Performance and Efficiency

In tests on the Qwen2.5-VL model, HAWK maintains 96.0% of the original accuracy after pruning 80.2% of visual tokens; the end-to-end inference time is reduced to 74.4% of the original; and GPU memory usage is significantly reduced, facilitating deployment on resource-constrained devices.

## Conclusion: Technical Significance and Industry Impact

Technically, HAWK provides an effective path to solve the computational efficiency bottleneck of MLLMs and reveals the phenomenon of functional differentiation of attention heads; industrially, the training-free feature is suitable for fast-iterating engineering environments, enabling rapid evaluation of deployment optimizations and shortening the productization cycle.

## Limitations and Future Outlook

Limitations: Currently, it only targets visual token pruning. Future directions: Expand to token optimization for other modalities; optimize the calculation of head importance weights; explore more aggressive compression schemes by combining technologies such as quantization and knowledge distillation.