ZINC: A High-Performance LLM Inference Engine for AMD RDNA3/RDNA4 GPUs

ZINC (Zig INferenCe Engine) is an open-source LLM inference engine optimized for AMD RDNA3/RDNA4 architecture consumer GPUs, developed using the Zig language and Vulkan API. It addresses the problem that consumer AMD GPUs are excluded from the ROCm ecosystem and lack sufficient support from existing tools, providing vLLM-level continuous batching and paged KV cache capabilities to enable these hardware to run LLM inference tasks efficiently.

AMD RDNA3/RDNA4 series consumer GPUs (such as RX9070, Radeon AI PRO R9700) have excellent hardware specifications like 576+GB/s memory bandwidth and cooperative matrix operations, but have long been neglected in the AI inference field. Reasons include: ROCm does not support consumer GPUs; vLLM relies on ROCm and cannot run; the llama.cpp Vulkan backend lacks targeted optimizations (e.g., SPIR-V toolchain issues, no tensor parallelism). This leads to millions of these GPUs in desktops being idle for inference tasks.

ZINC's core idea is to fully utilize the hardware capabilities of AMD consumer GPUs, with key features including: 1. Hardware deep optimization: Wave64 scheduling, architecture-aware chunking, fused operations, aiming to achieve over 90% of theoretical memory bandwidth for matrix multiplication; 2. Production-grade batching: Supports continuous batching and paged KV cache, a single RX9070 XT can serve 4+ concurrent users; 3. TurboQuant KV compression: Reduces cache memory by 5x, accommodating more sessions; 4. OpenAI-compatible API: No complex driver stack required, ready to use out of the box.

ZINC's tech stack prioritizes performance and controllability: Zig language (0.15.2+), Vulkan API, SPIR-V shaders, Bun build tool. Build and run steps: 1. Clone the repository: git clone https://github.com/zolotukhin/zinc.git && cd zinc; 2. Build: zig build (Linux auto-compiles shaders, macOS skips; force compilation with zig build -Dshaders=true); 3. Run: ./zig-out/bin/zinc -m /path/to/model.gguf --prompt "...".

Performance optimization focuses on maximizing memory bandwidth: optimized access patterns, fused kernels, architecture-aware chunking. TurboQuant compression reduces KV cache memory by 5x. Comparison with existing solutions:

Features	ZINC	llama.cpp (Vulkan)	vLLM (ROCm)
RDNA4 support	✅ Native optimization	⚠️ Basic support	❌ Not supported
Continuous batching	✅	❌	✅
Paged KV cache	✅	❌	✅
Tensor parallelism	In development	❌	✅
OpenAI API	✅	❌	✅
Deployment complexity	Low	Low	High

Ideal scenarios: 1. Individual users with AMD RDNA3/RDNA4 GPUs; 2. Small deployments (single/several consumer GPUs); 3. Budget-sensitive applications; 4. Edge computing in non-CUDA environments. Current limitations: Only supports Linux; mainly supports GGUF model format; some advanced features are still under development.

ZINC is hosted on GitHub under the MIT license, allowing free commercial use. The project has a dedicated website (zolotukhin.ai/zinc) and uses GitHub Actions for CI testing. Future outlook: Support for more GPU architectures, improved distributed inference, broader model format support, and active community contributions.

ZINC fills the gap in LLM inference for AMD RDNA3/RDNA4 consumer GPUs. Through deep optimization, production-grade batching, and simple deployment, it reactivates these neglected hardware. It lowers the hardware threshold for AI deployment, promotes ecosystem diversity, and embodies the open-source spirit. For AMD GPU users or those needing non-CUDA solutions, it is worth trying.