Building Spiking Neural Networks from Scratch: From Single Neurons to Modern Hopfield Networks

This project, published by Geomaniac15 on GitHub, implements a complete spiking neural network system from scratch using basic tools like NumPy and Matplotlib. It covers single neuron models (LIF, Hodgkin-Huxley), synaptic transmission, excitatory-inhibitory networks, STDP learning rules, Hopfield memory networks, and reveals the deep connection between these and Transformer attention mechanisms, providing learners with a complete knowledge path from biophysics to modern AI.

• Original Author/Maintainer: Geomaniac15
• Source Platform: GitHub
• Project Name: LIF-neuron-model
• Project Link: https://github.com/Geomaniac15/LIF-neuron-model
• Release Date: 2026-05-31

Project Vision: Starting from first principles, understand the computational basis of biological neural circuits and their connection to modern AI. Implement everything from scratch without relying on any deep learning frameworks, and build a knowledge arc from single neuron biophysics to modern Hopfield networks and Transformer attention mechanisms.

1. Single Neuron Models:

   • Leaky Integrate-and-Fire (LIF) neuron: Based on the differential equation τ_m dV/dt = -(V - V_rest) + R·I(t), simulates membrane potential integration and leakage, fires spikes when exceeding the threshold, and enters a refractory period.
   • Hodgkin-Huxley model: Includes voltage-gated Na+ and K+ ion channels, simulates action potential generation and ion dynamics via conductance equations.

2. Network Models:

   • Two-neuron synaptic transmission: Unidirectional connection, synaptic current uses an exponential decay kernel to simulate neurotransmitter clearance.
   • 100-neuron excitatory-inhibitory (E-I) network: 80 excitatory / 20 inhibitory neurons, randomly sparse connections, conductance-based synapses, STDP learning rules.
   • STDP associative memory network: 20 input /10 output neurons, lateral inhibition (winner-takes-all), adjusts feedforward weights via STDP.
   • Hopfield network:
     • Binary version: Hebbian weight learning, sign update rule, theoretical capacity ~0.14N.
     • Continuous version: Replaces sign update with softmax, mathematically equivalent to Transformer attention mechanism (Attention(Q,K,V) = softmax(QK^T/√d)·V).

3. MNIST Application: Uses methods like Top-k attention, prototype memory, and momentum update to implement digit completion tasks.

• LIF Neuron: Firing frequency ~36Hz at 2nA input, critical current is 1.5nA (no firing below this).
• Two Neurons: Neuron B fires ~10ms after Neuron A's spike, verifying causal synaptic transmission.
• E-I Network: Emerges 15Hz rhythmic oscillations; synchronous/asynchronous state transitions simulate differences between epileptic and healthy cortical activity.
• Hodgkin-Huxley Model: Generates real action potentials (peak +40mV, width ~3ms), with natural emergence of refractory periods.
• STDP Network: After training, can recognize damaged patterns with 20% bit flips, achieving pattern separation.
• Hopfield Network:
  • Binary version: Capacity ~14 patterns when N=100, perfect retrieval under 30% noise.
  • Continuous version: Capacity increased to 500+ patterns, near-perfect retrieval.
• MNIST Completion: Classification accuracy 57.2% under 30% noise, pixel overlap 89.6%; digits 0/1/7 are most reliable, while 2/3/8 are easily confused.

This project is a rare example of a complete spiking neural network implemented from scratch, without relying on frameworks like PyTorch/TensorFlow. It manually builds everything from single neurons to modern Hopfield networks using NumPy. Its value lies in:

1. Biological Inspiration: Drawing inspiration from real neuron behavior to understand the essence of neural computation.
2. Mathematical Foundations: Deeply grasping the physical and mathematical principles behind each model.
3. Step-by-Step Construction: From simple to complex, verifying understanding at each step.
4. Cross-Domain Connections: Revealing the deep links between classical neural networks and modern Transformers.

For learners who want to deeply understand the principles of neural networks, it is an excellent resource that helps go beyond API calls and truly master the core of neural computation.