Fast-Slow Learning: A Dual-Speed Mechanism for Enabling Continuous Adaptation in Large Language Models

This article introduces a framework called Fast-Slow Learning, which aims to resolve the core contradictions in the continuous learning of large language models. The framework treats model parameters as 'slow weights' (storing general knowledge with low update frequency) and optimized contexts as 'fast weights' (quickly adapting to specific tasks with frequent updates). Through the dual-speed mechanism, the model can quickly adapt to tasks while retaining general reasoning capabilities, improving sample efficiency by 3x and significantly reducing catastrophic forgetting.

There are two traditional ways for large language models to adapt to downstream tasks: parameter update (slow learning) and in-context learning (fast learning). Parameter update can deeply absorb task information but easily leads to catastrophic forgetting and reduced plasticity; in-context learning is fast and simple but has a low performance ceiling and is limited by the context window. Inspired by the human cognitive dual-system theory (System 1: fast intuition, System 2: slow rationality), researchers proposed the dual-speed learning mechanism.

The core of the Fast-Slow Learning framework is the collaboration between slow weights and fast weights: slow weights (model parameters) store general knowledge and remain stable; fast weights (optimized contexts) absorb task-specific information and are updated frequently. The Fast-Slow Training (FST) that implements this framework uses an alternating optimization strategy: first fix the slow weights to optimize the fast weights, then update the slow weights based on the performance of the fast weights, and prevent forgetting through KL divergence constraints.

Experimental results show: FST's sample efficiency is 1/3 of pure reinforcement learning; it has a higher performance ceiling; the model's deviation from the original distribution is 70% lower than pure reinforcement learning, reducing catastrophic forgetting; it has stronger adaptability to subsequent tasks in continuous learning, avoiding stagnation.

Fast-Slow Learning corresponds to the human cognitive dual-system theory: fast weights are similar to System 1 (fast response, limited processing depth), and slow weights are similar to System 2 (deep thinking, knowledge accumulation). The knowledge learned by fast weights is 'internalized' through slow weight updates, just like the process of human skills moving from conscious control to automation.

Application scenarios of Fast-Slow Learning include: personalized assistants (quickly adapting to user preferences while retaining general capabilities), professional tools (mastering specific norms without losing general knowledge), and continuous learning (fast weights update user feedback in real time, slow weights consolidate improvements regularly).

Current limitations: Fast weight optimization requires a certain number of samples, making convergence difficult in extreme few-shot scenarios; the interaction mechanism between slow and fast weights can be optimized; expansion directions: exploring medium-speed learning mechanisms (such as dynamic structure adjustment, memory module updates).

The Fast-Slow Learning framework balances efficiency and stability, providing an elegant solution for the continuous adaptation of large language models. It not only contributes practical technology but also demonstrates the value of interdisciplinary thinking. As the application of large models expands, systems that can continuously learn, adapt quickly, and not forget are more important, and Fast-Slow Learning has taken a key step in this direction.