Bidirectional LSTM and Attention Mechanism: Building a More Accurate Online Toxicity Detection System

This article deeply explores how to use bidirectional LSTM (BiLSTM) combined with attention mechanism, comparing with traditional feedforward neural networks (FFNN), to achieve significant performance improvement in the task of online toxicity comment classification. The content covers core points such as background challenges, technical solution comparison, experimental results, application significance, and future directions.

With the booming development of social media and online platforms, online toxicity content has become an increasingly severe social problem, which not only affects user experience but also may cause psychological harm. Traditional detection methods rely on keyword matching or shallow models, making it difficult to capture complex semantics and context dependencies; multi-label classification tasks (such as the coexistence of multiple types like insults, threats, etc.) are more challenging, and the ambiguity, sarcastic expressions, and context dependencies of natural language further increase the difficulty of automated detection.

Feedforward Neural Network (FFNN) Baseline

FFNN receives word embedding vector input and outputs classification results through fully connected layers. Its advantages are simple structure, fast training, and few parameters, but due to fixed-length input, it cannot capture word order and long-distance dependencies, which affects the performance of toxicity detection.

Bidirectional LSTM and Attention Mechanism

BiLSTM solves the gradient vanishing problem through gating mechanisms, and bidirectional training uses both past and future contexts; the attention mechanism dynamically assigns weights, focusing on key information (such as the role of the word "idiot" in a sentence for toxicity judgment), improving model performance and interpretability.

This study uses the Jigsaw Toxic Comment Classification benchmark dataset, which includes six categories: toxic, severe toxic, obscene, threat, insult, and identity hate. For the multi-label classification task, metrics such as precision, recall, F1 score, and AUC-ROC are used to comprehensively evaluate model performance.

Experimental results show that BiLSTM + attention mechanism significantly outperforms the FFNN baseline, better capturing sequence features and context dependencies, with higher accuracy and F1 scores for various toxicity types; the attention mechanism also enhances model interpretability—through visualizing weights, decision-making keywords can be directly seen, helping to understand model behavior and build user trust.

This study verifies the superiority of deep learning architectures in complex NLP tasks, providing a basis for platform content moderation technology selection; the interpretability of the attention mechanism helps reviewers quickly understand the basis for judgments, improving efficiency; the comparative experiment methodology provides optimization ideas for production-level system deployment.

In the future, we can explore the application of Transformer-based pre-trained models (such as BERT, RoBERTa), using self-attention mechanisms to further improve detection accuracy; multilingual toxicity detection is also an important direction—through transfer learning, knowledge from English models can be transferred to low-resource languages to meet the needs of global platforms.