QIG: A Fine-Grained Post-Training Quantization Method for Large Vision-Language Models Based on Quantization-Aware Integrated Gradients

This article introduces QIG, a CVPR 2026 paper, which is a fine-grained post-training quantization technique for large vision-language models (LVLMs). This method addresses the scale challenges in LVLMs deployment via quantization-aware integrated gradients, reducing storage and computational overhead while preserving model performance, thus providing a practical solution for deploying multimodal models on edge devices.

Large vision-language models excel in tasks like image understanding and visual question answering, but their massive scale poses significant deployment challenges. Quantization is an important model compression technique, yet traditional post-training quantization faces unique issues when handling LVLMs: it needs to process both visual and text modalities simultaneously, and the complex cross-modal interaction mechanism makes simple quantization strategies prone to significant performance loss.

The core innovation of QIG is transferring integrated gradients (originally an interpretability technique for explaining neural network predictions) to the quantization domain, identifying and preserving key weights and activation values that have the greatest impact on model outputs. This method considers the cumulative effect of weight changes on outputs (a global perspective), better maintaining the overall behavior of the model—especially showing distinct advantages in cross-modal feature interaction.

QIG adopts a fine-grained quantization granularity, using differentiated quantization parameters for different layers, modalities, and channels: the visual encoder requires higher precision to retain fine-grained visual information, while the text encoder can be compressed more aggressively. Guided by quantization-aware integrated gradients, it automatically identifies key parts for optimal bit allocation. It also considers the special processing needs of LVLMs' unique architectures (e.g., projection layers and alignment modules).

Standard vision-language benchmark tests show that QIG still maintains excellent performance under extremely low bit settings. Compared with existing post-training quantization methods, it has lower accuracy loss at the same compression ratio, and in some tasks, it is close to the full-precision model. Especially in complex tasks requiring fine-grained visual understanding (such as detailed image description and multi-object relationship reasoning), it performs excellently—proving that the fine-grained strategy retains advanced visual understanding capabilities.

QIG provides a practical solution for deploying LVLMs on edge devices: its post-training quantization feature does not require original training data or expensive fine-tuning, lowering the deployment threshold; the fine-grained strategy balances compression ratio and performance. With the popularization of multimodal AI applications, efficient model compression technology is a key bridge between cutting-edge research and practical applications, and QIG has made valuable explorations.