Implementing Neural Networks on FPGA: Practical Exploration of Building Low-Latency AI Cores with Verilog

This project was completed by a Vietnamese developer team. It implements an 8-6-6-2 four-layer fully connected neural network inference core on the Altera Cyclone IV E series FPGA chip using pure Verilog hardware description language. The project aims to address the low-latency requirements of edge devices and real-time control systems. By offloading AI algorithms to the hardware layer, it eliminates software stack overhead and achieves microsecond-level inference latency. Core technologies include fixed-point arithmetic optimization, pipeline parallel design, and LUT approximation of activation functions, which are suitable for industrial control, edge inference, and other scenarios.

With the popularization of artificial intelligence applications, neural network deployment scenarios have become diverse. Cloud GPU clusters provide strong computing power, but latency has become a bottleneck in edge devices and real-time control systems. Traditional software implementations require multi-level processing and are difficult to meet microsecond-level response requirements. As a reconfigurable hardware platform, FPGA allows direct circuit definition, mapping neural network forward propagation to parallel digital logic, eliminating software stack overhead and enabling pipeline parallel processing.

This project was completed by a Vietnamese team. The goal is to implement a complete neural network inference core on the Altera Cyclone IV E FPGA, developed using pure Verilog without using existing IP or HLS tools. The network architecture is an 8-6-6-2 four-layer fully connected network: the input layer receives 8-dimensional features, passes through two hidden layers with 6 neurons each, and outputs 2-dimensional results. Its compact scale is suitable for resource-constrained FPGAs and can handle classic classification and regression tasks.

Trade-off in Fixed-Point Arithmetic

Fixed-point arithmetic is used to save logic resources. Data bit width is optimized to balance precision and resource consumption, and FPGA's built-in DSP modules are used to accelerate neuron multiply-accumulate operations.

Parallelism and Pipeline Balance

Each layer is designed as an independent processing stage. FIFO is used between layers to buffer data, enabling sample pipeline processing to improve throughput; intra-layer neuron parallel computing accelerates inference.

Implementation of Activation Functions

Lookup tables (LUT) combined with linear interpolation are used to approximate Sigmoid/ReLU functions. Under the premise of ensuring precision, nonlinear operations are simplified to table lookup and multiplication operations.

A complete verification environment was established:

• Unit Testing: Verify the correctness of weight loading, bias addition, and activation function calculation for individual neuron modules.
• Integration Testing: Check data transfer between layers and timing coordination to avoid pipeline bubbles or conflicts.
• System Testing: Compare hardware output with software references using standard datasets to quantify precision loss.
• Timing Analysis: Ensure the design runs stably at the target clock frequency and meets timing constraints.

Synthesis results on Cyclone IV E show that the latency of a single forward propagation reaches the microsecond level, which is several orders of magnitude lower than general-purpose processors. In terms of resource utilization, logic units, memory blocks, and DSP units are fully utilized. Through reasonable partitioning and sharing, complete functions are implemented within limited resources.

Suitable scenarios:

• Industrial Control: Real-time equipment status monitoring and millisecond-level abnormal response.
• Edge Inference: Independent AI inference in network-free environments.
• High-Frequency Trading: Ultra-low latency market data analysis and decision-making.
• Sensor Fusion: Real-time processing of multi-channel inputs and output of control signals.

The project demonstrates the value of hardware-software co-design: hardware layer implementation allows full control of the computing process and customized clock cycles. Although the development cycle is long, it is irreplaceable for latency-sensitive applications. For engineers and students, implementing a network with Verilog is an excellent learning path, which can establish an intuitive understanding of algorithm complexity and hardware costs.