Sparse Autoencoders Crack the Black Box of Large Models: A Deep Case Study on Mechanistic Interpretability

This article introduces the open-source project mech_interpretability_case_study. Using Sparse Autoencoder (SAE) technology to address the polysemanticity problem, it decomposes entangled neuron activations in large language models into interpretable single-semantic features and implements activation-guided intervention techniques without fine-tuning, providing a systematic methodology for mechanistic interpretability of large models.

Polysemanticity is a core challenge in the field of neural interpretability. The traditional view holds that a single neuron encodes a specific concept, but in reality, a single neuron often responds to multiple unrelated concepts (e.g., numbers, punctuation, grammatical structures) simultaneously, leading to entangled activations that form complex distributed representations—this is the deep-seated reason why large models are difficult to interpret.

The core idea of SAE is to learn latent single-semantic basis vectors and decompose entangled activations into sparse, interpretable features. The architecture uses overcomplete dictionary learning: the encoder maps residual stream activations (896 dimensions) to a larger latent space (28672 dimensions); L1 regularization encourages sparsity; the decoder reconstructs the original activations. Overcompleteness provides degrees of freedom, and sparsity constraints ensure interpretability.

The project's 5-stage experimental workflow:

1. Activation Collection: Collect residual stream activations from the 12th layer of Qwen2.5-0.5B using the FineWeb-Edu dataset (2 million tokens);
2. SAE Training: Composite loss function (MSE + L1 regularization + decoder norm constraint) with an L1 coefficient warm-up mechanism;
3. Quality Evaluation: Quantitative metrics such as FVE, sparsity, and the proportion of dead features;
4. Feature Interpretation: Analyze activation contexts to build a feature dictionary;
5. Activation Guidance: Inject specific feature vectors to intervene in model behavior and verify the semantic authenticity of features.

Engineering highlights of the project:

1. Modular Architecture: Independent modules for each stage (data collection, training, evaluation, interpretation, intervention);
2. Unified Configuration Management: config.py centrally manages hyperparameters and supports command-line overrides;
3. Experimental Tracking Integration: ClearML records metrics, hyperparameters, and model versions to facilitate collaborative reproduction.

Significance of the project:

1. From Black Box to Gray Box: SAE provides a mapping tool from the model's internal state to human concepts;
2. Causal Verification: Activation guidance achieves a leap from correlation to causal intervention;
3. Model Safety and Alignment: Understanding internal representations helps identify and intervene in harmful features;
4. Scalability: The methodology can be extended to larger-scale models. Conclusion: Lays a solid foundation for mechanistic interpretability research.

Current challenges: Ensuring features correspond to real semantics, translating feature understanding into overall behavior prediction, and maintaining effectiveness in large-scale models. Future exploration can focus on the similarities and differences of features across models of different architectures/scales. It is recommended that interested readers refer to the project's complete code and documentation as a high-quality resource for getting started with mechanistic interpretability.