# When Classical Control Theory Meets LLM Inference: An 11-Chapter Exploration of Latency, Queues, and GPUs

> A unique learning project applies classical control theory to LLM inference service systems. Through 11 chapters of iterative experiments—from simulation to real GPUs—it reveals the most important truth in control design: understanding the controlled object is more important than the control law itself.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-17T17:45:55.000Z
- 最近活动: 2026-05-17T17:50:35.410Z
- 热度: 154.9
- 关键词: 经典控制理论, LLM推理服务, 延迟控制, 队列管理, 级联控制, PI控制器, 系统辨识, GPU调度, TTFT优化, 负载均衡
- 页面链接: https://www.zingnex.cn/en/forum/thread/llm-gpu11
- Canonical: https://www.zingnex.cn/forum/thread/llm-gpu11
- Markdown 来源: floors_fallback

---

## Introduction: Core of Cross-Disciplinary Exploration Between Classical Control Theory and LLM Inference Services

This project applies classical control theory to LLM inference service systems. Through 11 chapters of iterative experiments (3 successful controllers, 8 failure cases) and a real GPU closed-loop TTFT regulator, it reveals the core truth: understanding the controlled object is more important than the control law itself. The project covers the entire process from simulation to the Modal cloud platform's NVIDIA T4 GPU.

## Project Background: Exploration Journey from Simulation to Real GPUs

The project starts with the problem of using classical control to manage LLM inference latency and queues, with tools chosen as MATLAB and Python. It gradually transitions from pure simulation to local Ollama deployment, vLLM Apple Silicon version, and finally to the Modal cloud platform's NVIDIA T4 GPU. Each chapter's structure: specific control architecture design → implementation in a specific environment → experimental result analysis → failure cause dissection.

## Success Cases: Effective Control Implementations

Successful chapters include: 1. Single-loop LQR + pole placement (simulation verifies feasibility); 2. Cascade control architecture (in simulation, the inner loop controls batch size to queue depth, and the outer loop controls queue reference value to p95 latency); 4. Ollama single-loop integral control (first success on real hardware; batch size controls TTFT via GPU concurrency); 9. Cascade control in low-level GPU batch processing factory (found correct abstraction layers on Modal T4: precise batch executor, real backlog queue, measured GPU service time); 11. Closed-loop TTFT controller (velocity-type PI; set points 200/350/500ms with deviation within ±5ms, suppressing load step disturbances).

## Failure Lessons: Key Cognitive Breakthroughs

Core lessons from the 8 failure chapters: Chapter 2a (integration should be based on measured values rather than model predictions); Chapter 3 (cascade inner loop adjusts non-existent queue variables); Chapter5 (verify correctness before relying on metrics); Chapter6 (hardware scheduling behavior affects control architecture effectiveness); Chapter7 (cloud environment abstraction layers introduce signal noise); Chapter8 (control law applied to wrong abstraction layer); Chapter10 (correct controlled variable is TTFT, actuator is arrival rate metering).

## Architecture Evolution and Toolchain

Architecture evolution: Chapters1-2 pure simulation (MATLAB→llm_plant.m); Chapters3-4 real hardware (MATLAB/Simulink→Ollama HTTP→Apple Silicon GPU); Chapter6 real queue server (MATLAB→queue_server.py→Ollama→CPU); Chapters7-9 Modal remote GPU (MATLAB→Modal wrapper/vLLM→NVIDIA GPU); Chapter11 closed-loop TTFT controller (run_load_step.py→Modal wrapper→vLLM/Qwen→T4; PI parameters kp=0.03, ki=0.002; cycle 0.1s). Toolchain: MATLAB R2024b+, Control System Toolbox, Python3.11+, Modal, Ollama, vLLM.

## Core Insights: Essence of Control and Conclusions

Core insight: The control law itself is never wrong; the problem lies in applying it to the wrong abstraction layer. High-level API abstractions of modern LLM services hide underlying signals (e.g., aggregated latency, invalid queue metrics). Key conclusions: Measurement precedes modeling—need to understand the system's real behavior through experiments; control engineering is a combination of science (mathematical models, theory) and art (abstraction layers, variable selection).

## Engineering and Research Implications

For LLM service developers: Expose underlying signals, provide backpressure mechanisms, verify metric correctness; For control engineers: First characterize the controlled object, select the correct abstraction layer, start with simple control; For researchers: Value of failure cases, interdisciplinary thinking, experiment-driven. Project code is reproducible, link: https://github.com/hari-vasudevan/llm-serving-control, blog: https://vasudevanhari.substack.com/.