emotion_vector: Reproducing Anthropic's Emotion Vector Research with Local Open-Source Models

Anthropic's research published last year found that large language models contain identifiable "emotion vectors"—specific activation patterns with causal effects. The open-source project emotion_vector allows researchers and developers to reproduce this research locally using open-source models (such as Llama, Qwen, Mistral, etc.), supporting functions like emotion vector extraction, causal intervention, and visual analysis, thus promoting the democratization of AI mechanistic interpretability research.

In 2024, the Anthropic team published a paper exploring emotional representations in large models using the "activation patching" technique. They found that emotion vectors exist inside models: enhancing or suppressing specific patterns changes the model's performance on emotional tasks (e.g., enhancing the "joy" vector makes outputs more positive). This research sparked discussions on the nature of emotional representations and opened a new direction for mechanistic interpretability exploration.

The project's mission is to democratize cutting-edge research by reproducing Anthropic's core experiments on open-source models. Core functions include:

1. Emotion vector extraction: Identify relevant activation directions when the model processes emotional text
2. Causal intervention: Change the intensity of emotion vectors via activation patching to observe output effects
3. Visual analysis: Project high-dimensional vectors into low-dimensional space to display geometric structures
4. Multi-model support: Compatible with open-source models like Llama, Qwen, Mistral, etc.

Activation patching is the core technology, with the following process:

1. Prepare source input (containing target emotion) and target input (neutral/other emotions)
2. Record the activation state of specific layers when the model processes the source input
3. Replace the activation at the corresponding position when processing the target input
4. Observe output changes to verify whether the activation carries emotional information (i.e., emotion vectors)

Advantages:

• Fully controllable: Freely modify parameters and experiment with different model layers
• Low cost: No API fees, suitable for iterative exploration
• Privacy protection: Process sensitive data locally
• Reproducibility: Open-source code ensures verifiable results

Challenges:

• Computational resources: A 7B model requires at least 16GB of GPU memory
• Model differences: Emotional representation patterns may vary across different open-source models
• Parameter tuning: Parameters like layer selection and intervention intensity need careful adjustment

Application scenarios of the project include:

1. Model safety: Identify representations related to harmful tendencies and develop alignment technologies
2. Affective computing: Build more empathetic dialogue systems
3. Creative writing: Guide the generation of content with specific emotional tones
4. Interpretability research: A window to understand the internal mechanisms of models
5. Educational tool: Help students understand internal representations of neural networks

Usage Method:

1. Install dependencies and download open-source models
2. Prepare an emotional text dataset (supports customization)
3. Run the vector extraction script to identify emotion directions
4. Use the intervention script to test the impact of vectors on outputs

Community Outlook:

• Expand support for multilingual/code models
• Develop efficient vector extraction algorithms
• Establish standardized evaluation benchmarks
• Integrate other interpretability techniques like probe classifiers