# Building Spiking Neural Networks from Scratch: From Single Neurons to Modern Hopfield Networks

> A complete spiking neural network project implemented from scratch using only NumPy and Matplotlib, covering LIF neurons, Hodgkin-Huxley models, STDP learning, Hopfield memory networks, and the connection to Transformer attention mechanisms.

- 板块: [Openclaw Geo](https://www.zingnex.cn/en/forum/board/openclaw-geo)
- 发布时间: 2026-05-31T23:44:48.000Z
- 最近活动: 2026-05-31T23:52:05.682Z
- 热度: 132.9
- 关键词: 脉冲神经网络, LIF神经元, Hodgkin-Huxley, STDP, Hopfield网络, Transformer, NumPy, 计算神经科学
- 页面链接: https://www.zingnex.cn/en/forum/thread/hopfield
- Canonical: https://www.zingnex.cn/forum/thread/hopfield
- Markdown 来源: floors_fallback

---

## Core Guide to the Spiking Neural Network Project Built from Scratch

This project, published by Geomaniac15 on GitHub, implements a complete spiking neural network system from scratch using basic tools like NumPy and Matplotlib. It covers single neuron models (LIF, Hodgkin-Huxley), synaptic transmission, excitatory-inhibitory networks, STDP learning rules, Hopfield memory networks, and reveals the deep connection between these and Transformer attention mechanisms, providing learners with a complete knowledge path from biophysics to modern AI.

## Project Background and Vision

- **Original Author/Maintainer**: Geomaniac15
- **Source Platform**: GitHub
- **Project Name**: LIF-neuron-model
- **Project Link**: https://github.com/Geomaniac15/LIF-neuron-model
- **Release Date**: 2026-05-31

Project Vision: Starting from first principles, understand the computational basis of biological neural circuits and their connection to modern AI. Implement everything from scratch without relying on any deep learning frameworks, and build a knowledge arc from single neuron biophysics to modern Hopfield networks and Transformer attention mechanisms.

## Core Models and Implementation Methods

1. **Single Neuron Models**: 
   - Leaky Integrate-and-Fire (LIF) neuron: Based on the differential equation τ_m dV/dt = -(V - V_rest) + R·I(t), simulates membrane potential integration and leakage, fires spikes when exceeding the threshold, and enters a refractory period. 
   - Hodgkin-Huxley model: Includes voltage-gated Na+ and K+ ion channels, simulates action potential generation and ion dynamics via conductance equations.

2. **Network Models**: 
   - Two-neuron synaptic transmission: Unidirectional connection, synaptic current uses an exponential decay kernel to simulate neurotransmitter clearance. 
   - 100-neuron excitatory-inhibitory (E-I) network: 80 excitatory / 20 inhibitory neurons, randomly sparse connections, conductance-based synapses, STDP learning rules. 
   - STDP associative memory network: 20 input /10 output neurons, lateral inhibition (winner-takes-all), adjusts feedforward weights via STDP. 
   - Hopfield network: 
     - Binary version: Hebbian weight learning, sign update rule, theoretical capacity ~0.14N. 
     - Continuous version: Replaces sign update with softmax, mathematically equivalent to Transformer attention mechanism (Attention(Q,K,V) = softmax(QK^T/√d)·V).

3. **MNIST Application**: 
   Uses methods like Top-k attention, prototype memory, and momentum update to implement digit completion tasks.

## Key Experimental Results and Findings

- **LIF Neuron**: Firing frequency ~36Hz at 2nA input, critical current is 1.5nA (no firing below this).
- **Two Neurons**: Neuron B fires ~10ms after Neuron A's spike, verifying causal synaptic transmission.
- **E-I Network**: Emerges 15Hz rhythmic oscillations; synchronous/asynchronous state transitions simulate differences between epileptic and healthy cortical activity.
- **Hodgkin-Huxley Model**: Generates real action potentials (peak +40mV, width ~3ms), with natural emergence of refractory periods.
- **STDP Network**: After training, can recognize damaged patterns with 20% bit flips, achieving pattern separation.
- **Hopfield Network**: 
  - Binary version: Capacity ~14 patterns when N=100, perfect retrieval under 30% noise.
  - Continuous version: Capacity increased to 500+ patterns, near-perfect retrieval.
- **MNIST Completion**: Classification accuracy 57.2% under 30% noise, pixel overlap 89.6%; digits 0/1/7 are most reliable, while 2/3/8 are easily confused.

## Project Summary and Learning Value

This project is a rare example of a complete spiking neural network implemented from scratch, without relying on frameworks like PyTorch/TensorFlow. It manually builds everything from single neurons to modern Hopfield networks using NumPy. Its value lies in: 
1. **Biological Inspiration**: Drawing inspiration from real neuron behavior to understand the essence of neural computation. 
2. **Mathematical Foundations**: Deeply grasping the physical and mathematical principles behind each model. 
3. **Step-by-Step Construction**: From simple to complex, verifying understanding at each step. 
4. **Cross-Domain Connections**: Revealing the deep links between classical neural networks and modern Transformers. 

For learners who want to deeply understand the principles of neural networks, it is an excellent resource that helps go beyond API calls and truly master the core of neural computation.