AI Persuasion Benchmark Study: How Large Language Models Use Rhetorical Techniques to Influence Humans

This study focuses on mainstream large language models such as Claude, Gemini, and GPT. Through empirical analysis, it reveals the mechanisms by which these models use classical rhetorical techniques (logos: rationality, pathos: emotion, ethos: character) and modern influence strategies (scarcity, authority) to affect humans. The research aims to understand the boundaries of AI persuasion behavior and provide a basis for evaluating AI safety and ethics.

With the widespread application of large language models in daily life, whether AI has the ability to persuade humans and how it uses rhetorical techniques have become key issues. The "AI Persuasion Benchmark" project systematically tests mainstream models and analyzes persuasion strategies in different scenarios. Its results are crucial for evaluating the safety and ethical boundaries of AI in decision-making participation.

Multi-dimensional Prompt Classification

The study classifies test scenarios into four categories: entertainment, niche interests, ethical controversies, and factual errors, to observe strategy differences in different contexts.

Persuasion Technique Analysis Framework

Combining the three elements of classical rhetoric (logos: rational appeal, pathos: emotional appeal, ethos: character appeal) with modern influence strategies (scarcity, authority), it comprehensively evaluates AI persuasion behavior.

Model Strategy Differences

Different models show significant differences in their preferences for persuasion techniques, reflecting variations in training data, alignment strategies, etc.

Context Sensitivity

Model strategies adjust according to prompt categories: more relaxed in entertainment scenarios, while some models show intentions to refuse or correct in ethical controversy/factual error scenarios.

Potential Risks

Paying attention to the persuasion methods of AI in ethical controversy and factual error scenarios (educational vs. manipulative) and whether they reinforce misperceptions is of great significance for understanding safety boundaries.

Contribution to AI Safety

Provides empirical data to help design safe alignment strategies, identify social engineering attack vectors, and formulate ethical guidelines.

Implications for Users

Reminds users to maintain critical thinking when interacting with AI and pay attention to subtle persuasive elements in content.

Guidance for Developers

Helps improve model alignment training, design transparent interfaces, and establish user education mechanisms.

Research Limitations

As a graduation project, it has limited sample size, subjective annotations, and insufficient consideration of cultural contexts.

Future Directions

Explore scenarios such as long-term interactive persuasion dynamics, personalized strategies, cross-cultural comparisons, and adversarial testing.

This study reveals that large models have the ability to use rhetorical techniques to influence humans, which not only reflects progress in AI communication but also brings risks of abuse. Transparent empirical analysis is key to building trust in AI, reminding us to maintain independent thinking and critical judgment while enjoying the convenience of technology.