IREE Optimization Experiments: Dynamic Shape Inference Optimization for LLMs like DeepSeek, Qwen, and Gemma

The PLC Lab at Chongqing University has open-sourced the iree-optimization project, which focuses on conducting dynamic shape optimization experiments for mainstream large language models (LLMs) such as DeepSeek, Qwen, and Gemma using the IREE compiler, exploring technical paths for efficiently running LLMs on edge devices. Based on the IREE framework, the project addresses the static compilation challenges caused by dynamic shapes in LLM inference, providing references for compiler optimization in LLM deployment.

LLM inference efficiency is a key bottleneck in AI application deployment. Optimization of traditional deep learning frameworks (PyTorch/TensorFlow) is limited by runtime overhead. IREE is an MLIR-based compiler framework open-sourced by Google, supporting multiple hardware backends (CPU, GPU, etc.) and advanced compilation optimization technologies. However, LLM inference faces dynamic shape challenges: variable sequence lengths, autoregressive generation, batch size changes, and KV cache growth, which are difficult to handle effectively with static compilation.

The iree-optimization project is maintained by PLC-CQU and includes test scripts for the following models:

• DeepSeek: Exploring compilation to IREE format and dynamic shape handling;
• Qwen: Researching Chinese processing and dynamic sequence compilation optimization;
• Gemma: Exploring efficient dynamic inference of lightweight models in IREE.

The project's technical exploration directions include:

1. Compilation flow: Model import → Shape analysis → Compilation configuration → Code generation → Runtime integration;
2. Dynamic shape handling strategies: Full dynamic, partial dynamic, multi-version compilation, dynamic batching;
3. Performance optimization: Memory planning, operator fusion, quantization support, parallel strategies.

The value of this project lies in:

• Verifying the feasibility of deploying LLMs with the IREE compiler and providing reference implementations;
• Accumulating best practices for dynamic shape handling;
• Supporting cross-platform deployment (from server GPUs to mobile chips);
• Providing an academic research platform for fields such as compiler optimization.

Developers can use the project in the following ways:

1. Clone the repository to get test scripts and configurations;
2. Install the IREE compiler and related tools;
3. Run experiments to observe compilation and runtime results;
4. Adjust parameters to adapt to target hardware;
5. Submit Issues or PRs to contribute improvements.

The iree-optimization project represents the academic community's active exploration of LLM deployment optimization, applying advanced compiler technologies to mainstream LLMs and providing new possibilities for efficient and flexible inference. For developers deepening into LLM compilation optimization, it is an open-source project worth paying attention to and participating in.