Fast-Slow Training (FST) Framework: Enabling Continuous Adaptive Evolution of Large Language Models

Institutions including the University of California, Berkeley, proposed the Fast-Slow Training (FST) framework, which treats model parameters as "slow weights" (to maintain general reasoning abilities) and optimized context as "fast weights" (to absorb task-specific information). It achieves task specialization while preserving general capabilities. Experiments show a 3x improvement in sample efficiency, a 70% reduction in KL divergence, significantly better performance than traditional RL in continuous learning, and effective mitigation of catastrophic forgetting and loss of plasticity.

Traditional LLM training relies on parameter updates (e.g., RL), which easily leads to catastrophic forgetting and loss of plasticity; while in-context learning has low cost and fast adaptation, its performance ceiling is insufficient. Core question: Does learning have to be limited to the binary choice between "in-context" or "in-weight"?

Slow Weights

Corresponds to the actual model parameters, kept close to the pre-trained state to preserve general capabilities and avoid excessive drift.

Fast Weights

Virtual weights implemented via optimized context, learning task information from text feedback without modifying model parameters.

Collaboration Mechanism

Fast weights adapt quickly to tasks, slow weights maintain general capabilities; the division of labor balances efficiency and generalization.

• Sample efficiency is 3x higher than traditional RL, reaching target performance faster with the same data;
• KL divergence reduced by 70%, the model is closer to the original distribution and retains general knowledge;
• Significantly mitigates catastrophic forgetting, with little interference to old knowledge when learning new tasks;
• Maintains plasticity, making it easier to adapt to subsequent tasks after completing a task.

In dynamic continuous learning, traditional RL tends to have performance stagnation, while FST can continuously acquire new task knowledge, making it suitable for long-term deployment and practical applications that need to adapt to new environments continuously.

FST breaks the binary opposition between "parameter update vs. in-context learning" and provides a new training paradigm. Application value:

• Efficient fine-tuning: adapting to specific domains with little data;
• Stable deployment: maintaining general capabilities in continuous services;
• Multi-task adaptation: no interference when switching between tasks.

Inspired by the dual-process theory of human cognition (System 1/2 thinking), it rethinks the essence of learning. Future directions can include exploring multi-level learning mechanisms, extending to multimodal models and embodied intelligence; improving efficiency and adaptability without increasing parameters is an important direction in the LLM field.