230 Nanosecond Extreme Inference: FPGA Binary Neural Network Accelerator for High-Frequency Trading

This article presents an open-source project that implements an ultra-low-latency machine learning inference system on the Renesas SLG47910V FPGA. Using Binary Neural Networks (BNN) and XNOR-popcount logic, the inference time of a 16×64×3 network is reduced to 230 nanoseconds (23 clock cycles @100MHz), providing hardware-level real-time decision-making capabilities for high-frequency trading scenarios. The project adopts a three-layer heterogeneous architecture to achieve efficient collaboration between training, data processing, and inference.

In the field of High-Frequency Trading (HFT), every microsecond difference can affect profits and losses of millions of dollars. Traditional ML inference relies on GPUs or the cloud, with latency reaching milliseconds, which cannot meet sub-microsecond response requirements. This project uses BNN and FPGA acceleration to reduce inference latency to 230 nanoseconds—equivalent to the time light travels less than 50 meters in an optical fiber.

The project builds a three-layer system:

1. Python training environment: Train BNN using Larq and TensorFlow, with weights quantized to ±1;
2. ESP32-S3 firmware: Real-time ingestion of Binance market data, completing feature extraction and binary quantization;
3. FPGA inference core: Renesas SLG47910V executes inference, communicating with ESP32 via SPI. The core technology is replacing Multiply-Accumulate (MAC) with XNOR-popcount: When weights and inputs are quantized to ±1, multiplication is simplified to XNOR, and accumulation becomes popcount, eliminating floating-point multiplication with zero DSP usage. A time-multiplexing strategy is used to complete inference in 23 cycles (deterministic latency).

FPGA Resources: Core execution time is 230ns (23 cycles @100MHz), end-to-end SPI latency is ~290ns, total parameter count is 1216 bits, BRAM usage is 3.7%, DSP usage is 0, and logic cell usage is ~25%. Cross-clock domain synchronization uses a closed-loop Toggle synchronizer to solve metastability issues. Firmware Optimization: ESP32-S3 uses a FreeRTOS dual-task architecture (ingestion task processes data, result task handles inference output) to decouple CPU and inference latency. Quantization thresholds are calibrated during training and then hard-coded to ensure consistency between training and inference.

Timing Validation: Timing converges at 100MHz, with a setup time margin of 1.110ns for the critical path, and a maximum frequency of 112.48MHz. Accuracy: Among 1800 out-of-distribution samples, the precision/recall for BUY is 40.4%/89.6%, for HOLD it's 90.2%/72.6%, and for SELL it's 81.2%/90.5%. Model labels are derived from input feature rules, reflecting the BNN's ability to hardware-implement rules. Validation Methods: C/Python feature equivalence verification (100% bit consistency), formal verification (SymbiYosys + SVA), and co-simulation (Python + Icarus Verilog).

Core Insights: Quantization is key to performance (1-bit quantization compresses storage by 32x and shifts computation to bit domains); hardware-software co-design ensures cross-boundary consistency; deterministic latency is more important than peak performance. Application Scenarios: Beyond HFT, it can be extended to ultra-low-latency edge inference scenarios such as industrial real-time control, network packet processing, sensor fusion, and low-latency audio processing.

Current Limitations: 1-bit quantization limits model expressive power; labels are derived from input features, so no new market patterns are discovered; low precision (40.4%) of BUY signals leads to high false positives. Improvement Directions: Explore multi-level quantization (2/4 bits) to balance latency and expressive power; integrate multiple BNNs for voting to improve accuracy; implement lightweight weight updates on FPGA (online learning); introduce more market microstructure features to enhance discrimination ability.