Evolution of Fake News Detection Technology: Comparative Experiments from Traditional Machine Learning to Transformer and Large Language Models

This project, based on 40,000 news records, systematically compares the performance of three technical routes—traditional machine learning (SVC, XGBoost, MLP), fine-tuned Transformer (DistilBERT), and large language model prompting—on fake news detection tasks. It fully presents the development trajectory of NLP technology from classical to cutting-edge, providing a reference for technology selection.

In the era of information explosion, the spread of fake news causes social problems, making AI automatic identification an important direction in NLP. Developer caemanuela released an open-source project on GitHub, which not only trains classifiers but also compares the performance of three generations of NLP technologies. Using over 40,000 news records and three Notebooks to implement different methods, it intuitively demonstrates the evolution of technology.

The project uses over 40,000 labeled news items (including title, body text, and true/false labels). Preprocessing steps include removing HTML/special characters, converting to lowercase, word segmentation, and stopword filtering. Traditional ML additionally extracts TF-IDF features (vocabulary size and n-gram affect performance).

Compares SVC (kernel method), XGBoost (gradient boosting tree), and MLP (shallow neural network); uses TF-IDF + statistical features (text length, punctuation density); after tuning, all achieve high accuracy, proving that classical methods are still competitive.

Selects DistilBERT (a lightweight version of BERT that retains 97% of performance and reduces parameter count by 40%), transfers general language capabilities to fake news detection via fine-tuning; uses learning rate warm-up + linear decay to prevent overfitting; context-aware representations outperform traditional TF-IDF with higher precision and recall.

No training/fine-tuning required; guides LLM judgment via prompts; attempts zero-shot, few-shot, and chain-of-thought prompts (chain-of-thought performs better); advantages are low deployment threshold and flexibility, limitations are high inference cost, unstable output, and sensitivity to prompts; performs surprisingly well in some scenarios but lags behind fine-tuning in consistency and cost.

Traditional ML relies on manual features, has low cost and interpretability but a ceiling in semantic capture; Transformer fine-tuning achieves performance breakthroughs but requires labeled data and GPUs; LLM prompting has a low threshold but high operational cost. Technology selection depends on the scenario: choose traditional for batch processing, Transformer fine-tuning for high precision, and LLM prompting for rapid validation.

The project provides a reference framework for technology selection, which needs to consider dimensions such as accuracy, latency, and cost; its open-source nature facilitates reuse and improvement, promoting technological progress; technology selection should be appropriate rather than just choosing the newest.