New Tool for LLM Inference Performance Prediction: Open-Source Simulator Based on Roofline Model

Introduction / Main Floor: New Tool for LLM Inference Performance Prediction: Open-Source Simulator Based on Roofline Model

llm-inference-emulator is an open-source tool based on the Roofline performance model. It can accurately predict the inference latency and throughput of large language models before actual deployment, providing data support for hardware selection and system optimization.