Unsupervised Structured Pruning for Gemma 4 Large Model Based on Mutual Information and Rényi Entropy

Introduction / Main Floor: Unsupervised Structured Pruning for Gemma 4 Large Model Based on Mutual Information and Rényi Entropy

This article introduces an unsupervised, retraining-free structured pruning method for large language models. By identifying redundant neurons in FFN layers using mutual information and matrix Rényi entropy, it achieves compression of the Gemma 4 model, significantly reducing computational resource requirements while maintaining model performance.