PocketLLM: Extreme Compression of Large Language Models via Meta-Networks

PocketLLM is a large model compression method based on meta-networks proposed by authors such as Ye Tian and Chengcheng Wang. By projecting LLM weights into a discrete latent space using an encoder-codebook-decoder architecture, it achieves nearly lossless performance at a 10x compression ratio. This work has been accepted by AAAI 2026, and the project is open-sourced on GitHub, providing a feasible solution for deploying large models on edge devices. The original sources are GitHub/arXiv, paper link: https://arxiv.org/abs/2511.17637, published in November 2025 (arXiv submission).

With the expansion of LLM parameter scales (from billions to hundreds of billions), storage and transmission challenges have become prominent. For example, a 7B parameter model stored in 16-bit precision requires 14GB, which is unbearable for edge devices. Traditional quantization and pruning methods have significant performance losses at extreme compression ratios: quantization is limited by precision, and pruning destroys structural knowledge. Therefore, there is a need for innovative methods with high compression ratios and performance preservation.

PocketLLM adopts a latent space compression paradigm, with three core components: 1. Encoder: Divides weights into small blocks and projects them into latent vectors via a lightweight network; 2. Compact codebook: Stores representative vectors and uses indices instead of floating-point weights (e.g., a codebook with 1024 entries only requires 10-bit indices); 3. Decoder: Maps indices back to the weight space during inference, which is lightweight and low-overhead.

On the Llama2-7B model, PocketLLM achieves 10x compression with negligible drop in downstream task accuracy. Compared to traditional INT4 quantization, it has better performance degradation at the same compression ratio. Perplexity remains consistent on the WikiText-2 and C4 datasets, and lm-evaluation-harness verifies the effectiveness of downstream tasks.

PocketLLM brings multiple benefits to edge deployment: 1. Storage efficiency: The 7B model is reduced from 14GB to 1.4GB, suitable for mainstream mobile phones; 2. Transmission convenience: Reduced size lowers bandwidth requirements; 3. Privacy protection: Local deployment eliminates the need to upload data; 4. Open-source support: GitHub provides complete scripts for easy reproduction and expansion.

Current limitations: Does not involve activation value and KV cache compression. Future directions: Explore combination with Mixture of Experts (MoE) architecture to further improve the deployability of large models.