Research on the Activation Mechanism of Emotional Concepts in Open-Source Large Language Models

Based on Anthropic's research framework, this article analyzes the emotional concept activation mechanisms of five open-source large language models (Qwen, Mistral, Falcon, Zephyr, and OpenChat) using a paired emotional detection method, revealing significant differences in emotional processing among different models. The study found that all models exhibit emotional polarization (higher activation levels for negative/high-arousal emotions), obvious differences in activation intensity between models, and the top three emotions are concentrated in negative/high-arousal types such as fear, love, and anger. These findings have practical guiding significance for model selection, bias mitigation, and prompt engineering optimization.

In recent years, large language models (LLMs) have made breakthroughs in language understanding and generation, but there is still controversy over whether models truly 'understand' emotional concepts and their internal representation methods. Anthropic's closed-source model research first systematically explored the mechanism of emotional concepts, and open-source community researcher MustafaMunir123 extended the framework to the open-source field to understand the impact of different architectures and training strategies on emotional representation. The core idea of the study is to quantify the alignment between the model's internal activation and the direction of emotional concepts, rather than claiming that the model 'feels' emotions.

The paired emotional detection technique is adopted, defining a directional concept space through five groups of opposing emotions (sadness vs joy, anger vs calm, fear vs confidence, love vs hate, anxiety vs relaxation). The technical process includes:

1. Preparation of a balanced sample set for emotion pairs;
2. Extraction of hidden state activations at specific token positions in all Transformer layers;
3. Calculation of the mean activation difference between the two sides of the emotion pair to construct a direction vector;
4. Support for global continuous layer segments or emotion-specific layer selection strategies;
5. Calibration of scores as percentages to ensure comparability;
6. Repeating the evaluation across models and comparing the results.

Five open-source instruction-tuned models are selected:

• Qwen 4B Instruct (Alibaba's lightweight multilingual model);
• Mistral7B Instruct (Europe's efficient attention model);
• Falcon7B Instruct (high-quality model from the UAE's TII Research Institute);
• Zephyr7B (dialogue-optimized model fine-tuned based on Mistral);
• OpenChat7B (model focusing on open dialogue capabilities). The models cover different parameter scales (4B-7B), training data, and methodologies. The experiments are run on Kaggle's Tesla T4x2 GPU environment to ensure reproducibility.

Emotional Polarization Phenomenon: In all models, the activation levels of negative/high-arousal emotions (such as sadness and anger) are high, while those of paired emotions (joy and calm) are low, which may reflect the distribution of training data or encoding preferences. Inter-model Differences: OpenChat7B has the strongest emotional polarization (anger:99.4%, anxiety:99.4%); Qwen4B has strong polarization but a lower anxiety level (87.2%); Mistral7B has the weakest polarization; Zephyr7B has high anxiety activation (94.9%) and a relatively high confidence value. Concentration of Top 3 Emotions: The top three emotions across models are negative/high-arousal types such as fear, love, and anger. No model includes joy or calm in the top three, indicating emotional bias in the models.

Significance: Theoretically, it challenges the assumption that 'all large models represent concepts in the same way', and paired detection provides more abundant information; practically, it guides model selection (e.g., choosing OpenChat for sensitive emotion scenarios), bias mitigation, and prompt optimization. Limitations: It is only a representation-level analysis, not a conscious experience; the scores are internal tendencies under experimental conditions; only five models are covered. Future Directions: Testing larger-scale models, multilingual analysis, dynamic emotion tracking, model editing interventions, and correlation with downstream tasks.