VitaLLM: A Mixed-Precision Accelerator for Ternary-Weight LLMs on Edge Devices

VitaLLM is a mixed-precision accelerator for ternary-weight LLMs on edge devices. It adopts a dual-core design (TINT: Multiplier-Free Ternary-Integer Projection and BoothFlex: Reusable Radix-4 Booth Data Path) combined with a predictive sparse attention mechanism. Under the 16nm process, it achieves a decoding speed of 72.46 tokens/s, a prefill time of 0.88 seconds, occupies 0.214 mm² area and 120KB on-chip memory, and solves the precision-efficiency trade-off problem in edge LLM deployment.

Deploying LLMs on edge devices faces a fundamental contradiction between precision and computational efficiency: traditional FP16/INT8 quantization still cannot meet resource-constrained requirements; ternary-weight models (e.g., BitNet b1.58) can maintain quality while reducing computational demands, but the lack of dedicated hardware accelerators limits their practical applications.

TINT Core: Handles matrix multiplication between ternary weights and integer activations. It leverages the {-1,0,+1} weight characteristics to eliminate multipliers, and achieves efficient computation through look-up tables and sign selection circuits, reducing area and power consumption. BoothFlex Core: Based on radix-4 Booth encoding, it supports INT8×INT8 standard attention computation and ternary-INT continuous computation. Dynamic configuration switching is enabled without array duplication, realizing resource sharing.

To address the O(n²) complexity of Transformer attention, the following methods are adopted:

1. Leading-One Proxy: Uses the position of the Leading-One in values as a proxy indicator for attention scores;
2. Comparison-Free Top-K Selector: Locates the top-K candidates through bit pattern analysis, reducing KV cache reads and lowering memory bandwidth and computation load.

Head-Level Pipelining: Enables time multiplexing between multi-head attention, maintaining throughput while controlling hardware costs; Absmax Quantization Barrier: Unifies cross-core interfaces, supports dynamic quantization parameter calculation and propagation, simplifies data exchange, and controls communication overhead.

16nm process, 1GHz frequency, 0.8V voltage; Key metrics under BitNet b1.58 model: 72.46 tokens/s decoding speed, 0.88s prefill (64 tokens), 0.214 mm² area, 120KB on-chip memory; Ablation experiments verify the effectiveness of sparse attention, dual-core architecture, and quantization barrier.

Mixed precision is the inevitable path for edge AI; dedicated architectures are more efficient than general-purpose accelerators in edge scenarios; deep integration of hardware-algorithm collaboration (e.g., sparsity) is required to unleash potential.

Limitations: Limited model support (mainly BitNet b1.58), sparsity of long sequences (>4K tokens) to be verified, insufficient optimization in the training phase; Future: Expand quantization schemes (binary/quaternary), explore dynamic precision switching, and study multi-task resource sharing strategies.