CLLM: A Bare-metal Unikernel Written in C to Free LLM Inference from OS Constraints

CLLM is a GitHub project maintained by cognisoc (released on June 6, 2026). It is a bare-metal Unikernel written in pure C, booting directly on hardware without a traditional operating system, and designed specifically for LLM inference services. Its core idea is to pursue extreme inference performance and resource efficiency by eliminating the OS layer.

Most LLM inference optimizations focus on model architecture, quantization, or distributed deployment, while CLLM approaches from the system layer and questions the necessity of an operating system. Unikernel is a technology that compiles applications and OS kernels into a single image, containing only necessary components. Its advantages for LLM inference include: boot time reduced from seconds to milliseconds, extremely low resource usage, minimized attack surface, and no user-mode/kernel-mode switching overhead.

Boot and Initialization Layer

Adopts the Multiboot specification: boot.S sets up the stack and serial port, kernel.c initializes the VGA terminal and serial I/O, and memory.c implements the heap allocator.

Network Protocol Stack

Supports PCI bus enumeration, Intel e1000 network card driver, built-in HTTP server, and REST API.

Inference Interface Layer

Designed an API interface compatible with llama.cpp, allowing reuse of the GGUF model format and seamless migration of client code.

Completed features: Multiboot-compatible kernel, custom libc implementation, PCI enumeration and e1000 driver, HTTP server, llama.cpp-compatible API v1. Future roadmap: Integrate llama.cpp inference engine, GPU passthrough support, streaming token generation, and vLLM optimization technology integration.

• Edge Computing: Suitable for resource-constrained devices (e.g., industrial IoT, in-vehicle systems), with memory usage compressed to tens of MB.
• High-Concurrency Inference: Millisecond-level cold start supports the "one kernel per request" isolation model for Serverless scenarios.
• Security-Critical Scenarios: Small codebase, easy to formalize verification, minimized attack surface.

1. Hardware Compatibility: Requires dedicated driver support, cannot be used out-of-the-box like Linux;
2. Debugging Difficulty: Relies on serial port output and GDB remote debugging;
3. Ecosystem Isolation: Cannot directly use existing software packages;
4. Development Threshold: Requires in-depth understanding of OS underlying principles.

CLLM represents a radical optimization idea: removing the OS layer instead of optimizing on top of it. Although not suitable for all scenarios, it provides a new direction for dedicated systems pursuing extreme performance. As LLM scales grow, inference cost optimization becomes increasingly important. CLLM demonstrates the potential of system-level optimization—combining applications and kernels may break through performance ceilings.