Uninvited Guest: Single-Method Safety Assessment Cannot Fully Detect Risks of Personality-Injected Large Language Models

The study found that two personality injection methods—prompt engineering and activation manipulation—exhibit completely different vulnerability patterns. Using only one method for testing will miss the model's main failure modes, so a multi-method comprehensive assessment is proposed. This article will analyze from aspects such as background, methodology, evidence, and conclusions.

Personality injection technology for large language models brings customization possibilities to AI applications (e.g., more patient customer service, more encouraging educational assistants), but it also poses safety risks (e.g., being "too helpful" makes it easy to be induced to generate harmful content). Current safety assessments mostly focus on prompt-based personality injection and ignore activation manipulation, an underlying technology, which may lead to serious safety blind spots.

The study covers 5568 experimental conditions with human judgments, involving 4 mainstream models (Llama-3.1-8B, Gemma-3-27B, Qwen3.5, DeepSeek-R1-Distill-Qwen-32B). Using the Big Five personality model as a framework, it tests both prompt injection and activation manipulation methods and records the Attack Success Rate (ASR).

• Prompt engineering: Vulnerability rankings are consistent across different architectures (correlation coefficient: 0.71-0.96), and risks are predictable across architectures;
• Activation manipulation: Vulnerability patterns are highly architecture-specific (Llama-3.1-8B is highly vulnerable, Gemma/Qwen are more sensitive to prompts), with no correlation to prompt risks;
• Prosocial personality paradox: In Llama-3.1-8B, the personality with high conscientiousness + high agreeableness (P12) is safe in prompt assessment, but its ASR reaches 81.8% under activation manipulation;
• Geometric perspective: In Llama, conscientiousness is inversely correlated with the refusal mechanism; enhancing conscientiousness weakens the safety defense line;
• Reasoning ability: Reasoning models have lower ASR, but architecture differences still exist; the key to safety lies in the quality of strategy recall and self-correction.

1. A single assessment method is insufficient; a comprehensive assessment using multiple methods (prompt, activation manipulation, etc.) is needed;
2. Architecture specificity cannot be ignored; assessments need to be customized;
3. Surface safety needs to be deeply explored; prosocial personalities may hide risks;
4. Personality injection design needs to jointly optimize personality expression and safety boundaries, and establish multi-layered defenses.

Practical Recommendations:

• Comprehensive assessment should include prompt injection, activation manipulation, and emerging technologies;
• Customize assessments for the deployed architecture; do not blindly generalize results;
• Continuously monitor and update assessment methods.

Future Research:

• Develop robust personality injection methods that resist multiple injection techniques;
• Explore the deep relationship between personality traits and safety mechanisms;
• Research real-time monitoring and intervention mechanisms;
• Extend to multi-modal scenarios.