Explainable AI Practice for Vietnamese Hate Speech Detection: Combining QLoRA Fine-tuning and Chain-of-Thought Prompting

This article introduces an open-source project for Vietnamese hate speech detection. By combining large language models (LLMs), QLoRA efficient fine-tuning technology, and chain-of-thought prompting engineering, it builds an explainable AI system. This system can not only classify hate speech but also extract reasoning bases and implicit statements, addressing the dual challenges of scarce resources and lack of transparency in black-box models for non-English content moderation.

The development of social media has made the spread of hate speech more convenient. Content moderation for non-English languages (such as Vietnamese) faces two major challenges: first, scarce language resources lead to insufficient training data; second, traditional black-box models are difficult to explain their decision-making reasons, affecting the transparency and fairness of content moderation. LLMs perform well in NLP tasks, but how to apply them to hate speech detection in specific languages and achieve explainability still needs exploration.

1. Base Model Selection: Adopt the Qwen2.5-3B model, which has excellent multilingual support, strong understanding of Asian languages, and balances performance and computational efficiency with 3B parameters; 2. Efficient Fine-tuning: Use QLoRA technology, reduce memory usage through 4-bit weight quantization, insert low-rank adapters to learn task knowledge, and train less than 1% of parameters to lower computational thresholds; 3. Chain-of-Thought Prompting: Guide the model to generate reasoning processes (such as analyzing offensive vocabulary, target groups, and hostility levels) before outputting classification results, enhancing explainability.

The system has three-layer output: 1. Hate Speech Classification: Binary (hate or not) or multi-class (subdivided into race, religion, gender, etc.) judgment; 2. Reasoning Basis Extraction: Present key evidence in natural language (e.g., "group attack", "exclusionary language"); 3. Implicit Statement Recognition: Analyze implicit expressions, reveal the potential malicious intent of the text, and help moderators fully understand risks.

1. Content Moderation Assistance: Provide marked results and judgment bases for human moderators, improving moderation efficiency and consistency; 2. Transparency and Trust-building: Show users the reasons for violations, reduce appeal disputes, and enhance platform trust; 3. Low-resource Language Demonstration: The technical route (LLM + efficient fine-tuning + chain-of-thought prompting) provides a reusable paradigm for content moderation in other low-resource languages.

Limitations: The model's performance depends on the quality of training data, and there may be blind spots for emerging internet slang or subcultural expressions; the reasoning bases are sometimes too general and not precise enough. Future Directions: Expand training data to cover more hate speech variants; explore multi-modal fusion (text + image + video); optimize explainability to generate more specific bases; study adversarial attack defense mechanisms.

This project combines large language models, QLoRA fine-tuning, and chain-of-thought prompting to implement an explainable AI system for Vietnamese hate speech detection. It has direct value for Vietnamese internet governance and also provides a reference for global low-resource language content security construction. We look forward to more explainable AI applications that curb the spread of hate speech while protecting freedom of speech, building a healthy and inclusive online community.