InfiniLoRA: A Decoupled Multi-LoRA Service System Breaking Through Service Bottlenecks Under MoE Architecture

Introduction / Main Floor: InfiniLoRA: A Decoupled Multi-LoRA Service System Breaking Through Service Bottlenecks Under MoE Architecture

InfiniLoRA achieves a 3.05x increase in request processing rate under strict latency constraints by decoupling LoRA execution from base model inference, introducing innovations such as shared LoRA servers, parallel-aware execution, and SLO-driven resource allocation, effectively solving the scalability issue of LoRA services under the MoE architecture.