MoDeGPT: A New Method for Large Language Model Compression Based on Modular Decomposition

MoDeGPT is a large language model compression technique based on modular decomposition proposed in an ICLR 2025 paper. Its core lies in splitting LLMs into relatively independent functional modules and adopting differentiated compression strategies based on the characteristics of each module. It significantly reduces model size while maintaining performance, solving the problem that traditional compression methods struggle to balance compression ratio and performance.

The scale of large language models is expanding rapidly (from 175 billion parameters in GPT-3 to trillions in GPT-4), leading to a surge in training and inference costs and deployment difficulties. Traditional compression methods such as pruning, quantization, and knowledge distillation can reduce size but often sacrifice performance, making it hard to achieve an ideal balance between compression ratio and capability.

The core insight of MoDeGPT is that LLMs are composed of multiple relatively independent functional modules. Its theoretical basis comes from the analysis of the Transformer architecture: early layers are responsible for lexical and syntactic extraction, middle layers handle semantic context, and deep layers focus on reasoning and generation. This functional differentiation supports modular decomposition, allowing optimal compression schemes to be designed for each module.

1. Module Identification and Division: Automatically identify groups of layers with similar functions as functional modules by analyzing inter-layer activation patterns, attention distributions, and gradient flows; 2. Differentiated Compression Strategy: Adopt aggressive pruning and quantization for early feature extraction modules (low sensitivity to precision), and conservative compression for deep reasoning modules (to retain reasoning ability); 3. Inter-module Coordination: Introduce lightweight adaptation layers to ensure smooth information flow between compressed modules and avoid performance degradation.

ICLR 2025 experiments show that MoDeGPT achieves 4x volume compression while maintaining accuracy close to the original model—key modules retain more parameters, and auxiliary modules are significantly compressed. Compared to traditional global pruning, it performs better at the same compression ratio because global methods ignore layer function differences, while MoDeGPT can adjust adaptively.

1. Mobile Deployment: Compress models with billions of parameters to hundreds of millions, supporting deployment on smartphones and tablets; 2. Edge Computing: Customizable compression—prioritize retaining key modules in resource-constrained scenarios; 3. Model Service Optimization: Reduce memory usage and loading speed, increase concurrent requests, and lower inference costs.

Limitations: Module identification requires additional computational overhead; the optimal module division strategy varies by model architecture, requiring targeted tuning. Future directions: Develop more efficient module identification algorithms and explore modular dynamic adjustment mechanisms.

MoDeGPT is an important breakthrough in the field of LLM compression, balancing compression ratio and performance. The cbacary open-source implementation provides core algorithms, easy-to-use APIs, and example code, offering an experimental platform for researchers and developers to support further exploration and optimization.