XtraMAC: An Efficient MAC Architecture for FPGA-based Mixed-Precision LLM Inference

XtraMAC is a novel MAC architecture that supports integer, floating-point, and mixed-precision operations through a unified integer mantissa product microarchitecture. It addresses the limitations of traditional MAC units designed for a single data type. On the AMD Xilinx U55c FPGA, it achieves a 1.4-2.0x increase in computational density and a 27-51% reduction in resource consumption, providing an efficient and flexible hardware acceleration solution for FPGA-based mixed-precision LLM inference.

The inference efficiency of Large Language Models (LLMs) is a key bottleneck in the AI field. Mixed-precision quantization technology can reduce computational and storage overhead while maintaining accuracy, but traditional MAC units are designed for a single data type and struggle to efficiently support mixed-precision operations. Although FPGAs offer programmable flexibility, existing solutions have three major limitations: fixed data type constraints, low resource sharing efficiency, and insufficient mixed-precision support, leading to low DSP resource utilization and limiting parallelism and throughput.

The core innovation of XtraMAC is its unified data type adaptive microarchitecture. The key insight is that all supported MAC formats can be decomposed into shared integer mantissa products plus lightweight sign and exponent processing. Core designs include: dynamic operand packing (flexible packing based on data types to maximize DSP utilization), lightweight sign/exponent processing (minimizing floating-point operation overhead), and constant latency and initiation interval (1-clock-cycle latency and initiation interval to ensure efficient pipelining).

Evaluations on the AMD Xilinx U55c FPGA show: 1.4-2.0x increase in computational density (more MAC units deployed with the same resources); 27-51% reduction in resource consumption (reductions in LUT, FF, and DSP resources); up to 1.9x improvement in energy efficiency (increased computations per watt); up to 1.2x improvement in speed (reduced end-to-end inference latency).

The implementation strategies of XtraMAC include: mantissa sharing strategy (floating-point operands are decomposed into mantissa, exponent, and sign; mantissas are shared as integer multiplication hardware); runtime dynamic configuration (programmable control logic supports precision switching without reconfiguring FPGA bitstreams); pipeline optimization (deep pipeline design with constant initiation interval to avoid bubbles).

XtraMAC has a wide range of application scenarios: edge AI devices (deploying LLMs under power and area constraints); data center acceleration (reducing operational costs and carbon footprint); adaptive inference systems (dynamically adjusting precision to balance efficiency and accuracy); research prototype platforms (supporting experiments with new quantization schemes).

XtraMAC has room for improvement: expanding the precision range (supporting more aggressive low-precision formats like INT4 or custom formats); integration with storage systems (building memory-optimized systems); multi-chip expansion (multi-chip solutions for ultra-large-scale models).

XtraMAC represents an important advancement in FPGA-accelerated LLM inference. Through its unified microarchitecture, it achieves significant improvements in computational density, resource efficiency, and energy efficiency, proving that hardware acceleration for mixed-precision inference can balance efficiency and flexibility. The open-source implementation (GitHub link: https://github.com/Xtra-Computing/XtraMAC) lowers the entry barrier, promotes community standardization and improvement, injects vitality into the AI hardware ecosystem, and facilitates the widespread application of large model technologies.