Frontier: A High-Precision Discrete Event Simulator for Modern LLM Inference Services

Frontier is a discrete event simulator tailored for modern LLM inference services, supporting runtime optimizations including PDD/AFD decoupled execution, CUDA Graphs, and speculative decoding. On a 16-GPU H800 test platform, its average throughput error is less than 4%, end-to-end latency error is reduced from 44.9% to 6.4%, and it can scale to thousands of GPUs. This simulator aims to provide "decision-level fidelity" to help system designers optimize cluster configurations and architecture choices.

Modern LLM inference services have evolved into highly complex distributed systems, adopting technologies like decoupled execution, multi-level parallelism, and dynamic batching. Emerging workloads (inference chains, agents, RL rollbacks) introduce stateful requests and complex dependencies. System designers face decision-making challenges such as GPU cluster configuration and batch size setting, but existing simulators are based on simplified monolithic replica abstractions and cannot accurately capture the dynamic characteristics of decoupled services, leading to excessive prediction errors that are difficult to guide practical decisions.

Frontier uses decoupled abstraction to model system architecture, explicitly distinguishing nodes such as Prefill, Decode, Attention, and FFN, and accurately capturing the computation, communication, and memory behaviors of each role. It supports PDD/AFD decoupled mode, CUDA Graphs (trading off construction cost and runtime savings), speculative decoding (simulating the draft model validation process), dynamic batching (evaluating throughput-latency trade-offs), and fully supports stateful requests (multi-turn KV cache reuse, inference chain dependencies, etc.).

In the validation on a 16-GPU H800 cluster, Frontier's average throughput prediction error is less than 4%; the end-to-end latency error is reduced from 44.9% (homogeneous deployment) and 51.7% (decoupled deployment) of traditional simulators to 6.4% and 2.6% respectively. In addition, this simulator can simulate thousands of GPUs on ordinary CPUs, with a single run time in minutes, supporting large-scale parameter scanning and optimization search.

Frontier's application scenarios include: SLA-driven Pareto frontier exploration (identifying optimal configurations that meet SLAs), heterogeneous decoupled allocation optimization (determining the optimal ratio of different node types), agent scheduling validation (avoiding performance traps), and RL post-training reconfiguration (guiding parallel strategies and checkpoint frequency settings).

Feature	Traditional Simulators	Frontier
Architecture Abstraction	Monolithic Replica	Decoupled Role Nodes
Communication Modeling	Average Latency Proxy	Explicit Communication Patterns
Memory Modeling	Static Capacity	Dynamic Allocation & Compression
Optimization Techniques	Simplified Assumptions	Accurate Mechanism Modeling
Stateful Requests	Not Supported	Fully Supported
Traditional simulators often underestimate the communication overhead of decoupled deployments, while Frontier provides a more reliable decision-making basis by explicitly modeling KV cache transmission and synchronization mechanisms.

Currently, Frontier mainly supports decoder-only models; support for encoder-decoder architectures and emerging models (such as Mamba, RWKV) is still under development; the modeling accuracy for complex network topologies (e.g., multi-rail Fat-Tree) needs to be improved. In the future, it will integrate power consumption models, introduce uncertainty quantification, and combine with automatic optimization tools to achieve end-to-end configuration optimization.