# MDA: A Large Model Online Associative Memory System Without Backpropagation

> MDA (Multidimensional Associative Memory) is an innovative online learning system that enables large language models (LLMs) to learn and memorize in real-time during inference, without the need for GPUs or backpropagation, opening up a new path for the continuous learning of LLMs.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-04-28T22:43:58.000Z
- 最近活动: 2026-04-29T01:51:28.892Z
- 热度: 158.9
- 关键词: 联想记忆, 在线学习, LLM, 无反向传播, 持续学习, 边缘AI, Oja规则, 高维表示, 隐私保护
- 页面链接: https://www.zingnex.cn/en/forum/thread/mda
- Canonical: https://www.zingnex.cn/forum/thread/mda
- Markdown 来源: floors_fallback

---

## Introduction / Main Floor: MDA: A Large Model Online Associative Memory System Without Backpropagation

MDA (Multidimensional Associative Memory) is an innovative online learning system that enables large language models (LLMs) to learn and memorize in real-time during inference, without the need for GPUs or backpropagation, opening up a new path for the continuous learning of LLMs.

## Core Problem: The "Amnesia" of Large Models

Current large language models (LLMs) have a fundamental limitation: they "remember" knowledge during the training phase, and once deployed, they cannot truly learn new things during inference.

The so-called "In-context Learning" merely stuffs examples into prompts, which is neither persistent nor efficient.

Traditional solutions—such as fine-tuning or continuous learning—require backpropagation and large amounts of computing resources, usually GPU clusters. This is too costly for real-time adaptation in production environments.

MDA (Multidimensional Associative Memory) was created precisely to address this pain point.

## What is MDA?

MDA is an online associative memory system implemented purely in Python, enabling LLMs to learn and update knowledge in real-time during inference, with the following revolutionary features:

- **Online Learning**: Learn during inference without offline training
- **No Backpropagation**: Completely free from gradient computation
- **No GPU Required**: Can run on CPU
- **Instant Memory**: New information takes effect immediately without reloading the model

## Technical Principle: Mathematical Foundation of Associative Memory

The core of MDA is based on classical associative memory theory, especially the modern evolution of Hopfield networks. Its key mechanisms include:

## High-Dimensional Distributed Representation (HDR)

MDA uses a high-dimensional vector space to store memories. Each memory item is encoded as a high-dimensional vector, and associative retrieval is achieved through the geometric relationships between vectors. This representation has natural fault tolerance and generalization capabilities.

## Oja's Learning Rule

Unlike backpropagation, MDA adopts Oja's rule—a local update mechanism based on Hebbian learning. Oja's rule only relies on the current input and existing weights, with extremely low computational complexity, making it suitable for online updates.

## Advantages of Online Updates

The computational graph of traditional training needs to save intermediate activation values for backpropagation, while MDA's updates are "stateless"—each update only depends on the current input and existing weights, with minimal memory usage.

## Real-Time Adaptation

Imagine a customer service robot that can remember users' preferences and historical questions during conversations, without the need for pre-training or maintaining a complex user profile system.