AI Persuasion Capability Benchmark: How Large Language Models Apply Classical and Modern Rhetorical Techniques

Core Guide to the AI Persuasion Capability Benchmark

This study systematically evaluates the persuasion capabilities of three mainstream large language models—Claude, Gemini, and GPT—in applying classical rhetoric (logos/rationality, pathos/emotion, ethos/credibility) and modern strategies (scarcity principle, authority effect) across different scenarios. The research aims to understand the current state, risks, and application value of AI persuasion capabilities, providing references for AI safety and responsible deployment.

Project Information: Original author adamli25-llooma, source from GitHub project ai-persuasion-benchmark, updated on June 16, 2026.

Research Background: Why Is AI Persuasion Capability Important?

Large language models are widely used in interactive scenarios such as customer service, education, and debate, where persuasion capability is one of their core requirements. However, this capability is a double-edged sword:

• Positive aspects: Facilitate knowledge dissemination, improve patient compliance, and explain product value;
• Risk aspects: May be used to spread misinformation, manipulate decisions, or bypass safety mechanisms.

Understanding AI persuasion capabilities and their strategy preferences is crucial for AI safety research and responsible deployment.

Research Design and Methods

Evaluation Dimensions

1. Persuasion techniques: Covering the three classical rhetorical appeals (logos/rationality, pathos/emotion, ethos/credibility) + modern strategies (scarcity, authority effect);
2. Prompt scenarios: Four categories (lighthearted/absurd, niche professional, ethically questionable, factual errors).

Test Models

• Claude series (Anthropic)
• Gemini series (Google)
• GPT series (OpenAI)

Research Paradigm

Adopting the prompt-response paradigm: Present various prompts to models, analyze persuasion strategies in responses, and compare behavioral differences between models.

Observation of Model Persuasion Strategy Preferences

Although full data is not disclosed, key dimensions can be inferred from the design:

1. Strategy diversity: Do models rely on fixed strategies or flexibly use multiple techniques?
2. Context sensitivity: Do they adjust strategies based on scenarios (e.g., using logos/ethos for professional topics, pathos for emotional topics)?
3. Ethical boundaries: When faced with ethically questionable/factual error prompts, do they refuse or "rationalize"?
4. Cross-model differences: Do persuasion behaviors of different models reflect differences in training objectives and safety strategies?

Technical Implementation and Project Structure

As a graduation project, the repository provides a reproducible evaluation framework:

• Prompt dataset: Classified and organized test prompts;
• Response collection module: Interfaces for batch获取 of model responses;
• Persuasion strategy annotation: Annotation scheme for identifying strategies in responses;
• Analysis scripts: Statistical analysis and visualization tools.

This open-source framework provides references for the AI safety community, helping to continuously monitor the evolution of model persuasion capabilities.

Application Value and Insights

For AI Developers

Improve safety training strategies to ensure models use persuasion capabilities in appropriate scenarios and exercise restraint in inappropriate ones.

For Deployers

When selecting models for high-risk scenarios, refer to their persuasion behavior characteristics to make informed decisions.

For Safety Researchers

Provide an actionable framework to continuously monitor changes in model persuasion capabilities.

For Policymakers

Understand the current state of AI persuasion capabilities to help formulate regulatory frameworks that balance innovation and safety.

Limitations and Future Directions

Current Limitations

As a graduation project, sample size, evaluation depth, and model coverage may be limited.

Future Expansion

1. Expand model range (including open-source and dedicated models);
2. Add more psychological influence strategies;
3. Design more refined effectiveness evaluation metrics;
4. Longitudinally track behavioral changes in model version iterations;
5. Explore intervention measures (training/prompt engineering) to guide persuasion behaviors.

Conclusion

The AI Persuasion Benchmark represents an emerging direction for the systematic evaluation of AI persuasion capabilities. As AI's role in human life grows, ensuring its "appropriate persuasion" (effective communication in legitimate scenarios, no abuse in inappropriate scenarios) is crucial. This study provides valuable tools and insights for this goal.