Atom NPU: An Open-Source Neural Network Accelerator for Qwen2 Large Model Inference

Atom NPU is an open-source Verilog hardware accelerator project designed specifically for Qwen2 large language model inference, including a complete Python golden model, test vector generator, and verification testbench. This project fills the gap in the open-source community where complete NPU implementations for large language model inference are scarce. It provides a full design flow from algorithm modeling to silicon implementation, and its systematic verification methodology reflects industrial-grade development standards, offering valuable references for hardware architecture research and edge AI deployment.

As the parameter scale of large language models (LLMs) continues to grow, the hardware efficiency of inference computation has become a key bottleneck. Cloud deployment relies on expensive GPU clusters, while edge device deployment faces dual constraints of computing power and power consumption. Dedicated neural network processing units (NPUs) as ASIC acceleration solutions can improve energy efficiency, but complete NPU implementations for LLM inference are relatively scarce in the open-source community. Most open-source hardware projects remain at the level of convolutional neural network accelerators, with limited support for operations unique to the Transformer architecture (such as attention mechanisms, LayerNorm, Softmax, etc.).

Atom NPU is optimized specifically for Qwen2 large language model inference, implemented using the Verilog hardware description language, and provides a full design flow from algorithm modeling to silicon implementation. Its notable feature is the systematic verification methodology: a supporting Python-implemented golden model is used to verify the numerical correctness of the hardware design. This hardware-software co-verification method is relatively rare in open-source hardware projects and reflects industrial-grade development standards.

The Atom NPU architecture is optimized for Transformer inference workloads and supports Qwen2's features such as Grouped Query Attention (GQA), Rotary Position Encoding (RoPE), and SwiGLU activation function. The project includes complete testing infrastructure: a test vector generator automatically generates input data for boundary conditions, and the testbench supports module-level and system-level functional verification; the Python golden model helps algorithm engineers verify computational logic before hardware implementation, also facilitating the exploration of quantization strategies and enabling the comparison of outputs to locate the source of numerical errors.

Atom NPU has reference value for multiple groups: hardware architecture researchers can analyze its microarchitecture design and performance trade-offs; chip design engineers can refer to key technologies for mapping models to hardware (such as operator decomposition, data rearrangement, pipeline scheduling, etc.) and the verification environment; edge AI developers can explore the possibility of running large models on resource-constrained devices (combining technologies like quantization and pruning). This project enriches the open-source AI hardware ecosystem, with its unique feature being end-to-end optimization for Qwen2, reflecting the trend of model-hardware co-design.

As a research-oriented open-source project, Atom NPU has limitations: it lags behind commercial NPUs in terms of toolchain completeness and compiler support; the coverage of actual hardware verification is insufficient (there may be differences between RTL simulation and real silicon operation). Future directions include: supporting more model architectures (such as Llama, Mistral), integrating advanced quantization schemes (such as GPTQ, AWQ), and developing supporting compiler toolchains to enable automated model deployment.

Atom NPU contributes a complete NPU design case for large language model inference to the open-source community. Its systematic verification methodology, dedicated optimization for Qwen2, and open code repository provide valuable references for hardware architecture research and edge AI deployment. As the demand for deploying large models on edge devices grows, such dedicated accelerator designs will play an increasingly important role.