Intrinsic Memory Mechanisms in Spiking Neural Networks: From Biological Inspiration to Hardware Implementation

Project Core Overview

This research project was developed by nkanderson (released on GitHub on 2026-05-26, link: https://github.com/nkanderson/intrinsic-memory-SNNs). It focuses on the intrinsic memory mechanisms in Spiking Neural Networks (SNNs), models the historical states of neurons using fractional-order differentiation, compares the performance and overhead of traditional LIF models in digital hardware (FPGA) implementations, and explores a biologically inspired path for low-power neuromorphic computing.

Research Background and Problem Statement

Traditional Artificial Neural Networks (ANNs) simplify the dynamic characteristics of biological neurons. As the third generation of neural networks, SNNs use spike signals to transmit information and are a key direction for low-power neuromorphic computing. The intrinsic memory of biological neurons (adjusting current responses based on historical activity) is an important feature of SNNs, but how to effectively implement this mechanism in digital hardware while balancing computational overhead and performance remains an open issue. Traditional LIF models ignore the long-term impact of neurons' historical activities and are difficult to capture biological plausibility.

Research Methods: Three-Layer Framework and Fractional-Order Modeling

The project adopts a three-stage research framework:

1. Design Space Analysis: Focuses on neuron history term representation, fractional-order differentiation implementation (replacing integer-order differentiation to introduce memory effects), and fixed-point precision vs. resource trade-offs;
2. Training and Simulation: Uses snnTorch to train SNNs, verifies via software simulation after quantizing the model, then performs hardware simulation using SystemVerilog and cocotb;
3. FPGA Benchmarking: Synthesizes the model onto an FPGA platform and evaluates resource usage and inference performance.

Fractional-order differentiation modeling: Replaces the integer-order derivative in the LIF model with a fractional-order derivative (0<α<1), whose definition includes the integral of historical states and naturally has memory characteristics: D^α V(t) = 1/Γ(1-α) * d/dt ∫[0 to t] (V(τ)/(t-τ)^α) dτ

Experimental Design and Validation Process

• Task Selection: Uses the classic cart-pole balance control task as a benchmark. The SNN controller needs to receive system states (position, velocity, angle, angular velocity) and output control decisions;
• Quantization Strategy: Performs fixed-point quantization on weights, activations (membrane potential/spike output), and historical states to optimize hardware deployment efficiency;
• Validation Method: Conducts hardware-software co-simulation via the cocotb framework to verify the correctness of the SystemVerilog implementation, then performs FPGA hardware testing.

Research Significance and Application Prospects

• Advancement of Neuromorphic Computing: Improves the biological plausibility of SNNs, explores low-power temporal information processing, and evaluates the hardware feasibility of complex models;
• Potential Applications: Temporal pattern recognition (gesture, speech), robot control (more stable state memory controllers), edge AI devices (complex temporal reasoning under resource constraints);
• Performance Comparison: Compares the resource usage and inference performance of the intrinsic memory model and the traditional LIF model on FPGA.

Technology Stack and Toolchain

Layer	Tool/Technology	Purpose
Deep Learning Framework	snnTorch	SNN training and simulation
Hardware Description Language	SystemVerilog	Digital circuit design
Verification Framework	cocotb	Python-based hardware testing
Deployment Platform	FPGA	Hardware prototype verification
Programming Language	Python	Training scripts and data analysis

Summary and Future Outlook

This project demonstrates the transformation of biological neuroscience insights into engineering models and provides a complete research paradigm from theoretical modeling to hardware verification. The application of fractional-order differentiation in neuron modeling represents a direction for neural networks to return to biological plausibility. In the future, as neuromorphic chip technology matures, SNNs with intrinsic memory are expected to play an important role in low-power edge computing scenarios.