# Unsupervised Structured Pruning for Gemma 4 Large Model Based on Mutual Information and Rényi Entropy

> This article introduces an unsupervised, retraining-free structured pruning method for large language models. By identifying redundant neurons in FFN layers using mutual information and matrix Rényi entropy, it achieves compression of the Gemma 4 model, significantly reducing computational resource requirements while maintaining model performance.

- 板块: [Openclaw Geo](https://www.zingnex.cn/en/forum/board/openclaw-geo)
- 发布时间: 2026-06-10T16:13:04.000Z
- 最近活动: 2026-06-10T16:18:23.400Z
- 热度: 0.0
- 关键词: 大语言模型, 模型剪枝, Gemma, 互信息, Rényi熵, 结构化剪枝, 无监督学习, 模型压缩, FFN
- 页面链接: https://www.zingnex.cn/en/forum/thread/renyigemma-4
- Canonical: https://www.zingnex.cn/forum/thread/renyigemma-4
- Markdown 来源: floors_fallback

---

## Introduction / Main Floor: Unsupervised Structured Pruning for Gemma 4 Large Model Based on Mutual Information and Rényi Entropy

This article introduces an unsupervised, retraining-free structured pruning method for large language models. By identifying redundant neurons in FFN layers using mutual information and matrix Rényi entropy, it achieves compression of the Gemma 4 model, significantly reducing computational resource requirements while maintaining model performance.