Fake News Detection System Based on TF-IDF and Logistic Regression: From Algorithm to Application

This project is a full-stack machine learning web application maintained by smrity-shreya on GitHub (Project link: https://github.com/smrity-shreya/Fake-news-detector, released on June 9, 2026). Its core function is to classify news articles as real or fake using TF-IDF vectorization and logistic regression algorithms, providing a complete training process, REST API interface, and user-friendly web interface, aiming to address the need for rapid fake news detection in the era of information explosion.

In the era of information explosion, fake news has become a serious social problem, affecting public perception and even causing social unrest. According to statistics, over 60% of internet users have encountered suspected fake news on social media, and the speed of manual verification is far behind the speed of information dissemination. Therefore, automated fake news detection systems have become an essential need, which can assist manual review or block obviously false information. This project is precisely a complete solution built to address this demand.

Technical Architecture: Adopts a full-stack architecture, with layers and technology stack as follows:

Layer	Technology Stack
Backend	Python + Flask
Machine Learning	Scikit-learn + Logistic Regression
NLP Processing	TF-IDF + NLTK Stop Words
Frontend	HTML + CSS + Bootstrap5

Core Algorithms:

1. TF-IDF: Converts text into numerical vectors. TF (Term Frequency) = number of times a word appears in a document / total number of words in the document; IDF (Inverse Document Frequency) = log(total number of documents / number of documents containing the word); TF-IDF value is the product of the two.
2. Logistic Regression: Reasons for selection include strong interpretability (clear feature weights), fast training, resource-friendly, and low overfitting tendency (when combined with regularization). Text Preprocessing: Tokenization → Stop Word Removal → Lowercase Conversion → Optional Stemming.

Training Process: Implemented via train_model.py: Data Loading (CSV) → Feature Extraction (TF-IDF) → Model Training → Performance Evaluation (Accuracy, Precision, Recall, F1 Score) → Model Saving. Web Interface: Input box (paste news title/body), quick submission (Ctrl/Cmd+Enter), result display (predicted category + confidence), history records (latest 20 entries). REST API:

• POST /predict: Receives JSON ({"text": "..."}), returns prediction result (REAL/FAKE), confidence, probability, etc.;
• GET /history: Returns latest 20 records;
• GET /health: Service status monitoring.

Recommended Datasets: LIAR (10,000+ political statements with fine-grained labels), FakeNewsNet (includes news, social context, and dissemination information). Optimization Directions:

1. Feature Engineering: Try n-gram (2-gram/3-gram) to capture phrase information;
2. Ensemble Learning: Combine multiple classifiers to improve robustness;
3. Deep Learning: For large-scale data, try pre-trained models like BERT;
4. Multimodal: Combine multi-source information such as title, body, and images.

Quick Deployment Start:

1. Install dependencies: pip install -r requirements.txt;
2. Prepare data: Place CSV file in dataset/news.csv (supports text columns: text/title/content; label columns: label/class/target, values are REAL/FAKE or 1/0);
3. Train model: python train_model.py;
4. Start service: python app.py;
5. Access: http://127.0.0.1:5000. Application Scenarios: Social media platforms (pre-publishing detection), news aggregation apps (filtering suspicious articles), personal user tools (assessing credibility), research analysis (batch analysis of dissemination patterns).

Current Limitations: Language limitation (mainly for English), domain sensitivity (specific domains require specialized training), adversarial examples (complex fake news may bypass detection). Ethical Considerations: The system may be abused (suppressing dissent, information censorship, truth monopoly), so it is necessary to: maintain transparency (publicize detection standards), provide an appeal mechanism, and combine manual review instead of full automation.

Fake news detection is a complex issue intertwined with technology and ethics. This project uses TF-IDF + logistic regression to build a practical detection system. Although it cannot solve all problems, it provides a feasible starting point for automatic information quality screening. Its completeness and scalability (from data processing to web interface) make it an excellent learning case for NLP application development beginners.