Hands-On Practice for a Machine Learning-Based SMS and Email Spam Detection System

This project aims to use Python and machine learning techniques to build a real-time classification system for identifying SMS and email spam, covering the complete workflow from text preprocessing, feature extraction, model training to Streamlit deployment. Core technologies include TF-IDF feature extraction, Naive Bayes and other classification algorithms. It achieves high-precision detection through an open-source system and provides a user-friendly interactive interface, solving the problem that traditional rule-based filtering struggles to handle spam variants.

In the mobile internet era, spam SMS (accounting for about 15% of daily global messages) and phishing emails (over 50% of corporate emails) trouble users. Traditional rule-based keyword filtering is hard to handle complex variants, so machine learning-based intelligent detection has become the mainstream. This project, SMS-Email-Spam-Classification, is an open-source system that achieves high-precision classification and provides a user-friendly interface via Streamlit, making it easy for non-technical users to use.

The core tech stack uses mature tools from the Python ecosystem: NLTK and spaCy for text processing (word segmentation, stopword removal, stemming); TF-IDF vectorization for feature engineering; comparing models like Naive Bayes, Logistic Regression, Random Forest, etc., and selecting the optimal one for deployment. The architecture follows the workflow: data collection and cleaning → preprocessing → feature engineering → model training and evaluation → persistence → Web deployment. The modular design ensures maintainability and scalability.

Preprocessing involves multiple cleaning steps: removing HTML tags, special characters, and URLs; converting to lowercase for uniform formatting; using NLTK stemming to reduce words to their root forms (e.g., running → run); removing stopwords (like 'the', 'is'). Based on the characteristics of SMS (short, colloquial, many abbreviations) and emails (formal, structurally complete), a differentiated processing strategy is implemented, with configurable parameters to adapt to different text types.

TF-IDF is used to convert text into numerical vectors, considering term frequency and inverse document frequency—rare but high-frequency words are of great value. N-gram features (bigram, trigram) are explored to capture contextual information (e.g., the phrase 'free claim'), improving classification accuracy.

Compare Naive Bayes (baseline), Logistic Regression (interpretability), Random Forest/Gradient Boosting Trees (ensemble stability). Evaluation focuses on precision and recall, balancing missed detections (spam misclassified as normal) and false positives (normal misclassified as spam), and adapting to different business scenarios by adjusting thresholds and weights.

A Web interface is built using Streamlit. Users input content and click to predict, getting results in seconds with confidence scores displayed. Streamlit's advantages: pure Python development without front-end knowledge needed; model serialization (pickle/joblib) for loading; local processing to protect privacy.

This project demonstrates end-to-end machine learning practice. It serves as a text classification case for beginners and provides a modular code starting point for developers. Future expansion directions: introducing BERT deep learning, multilingual detection, active learning to optimize models, and developing browser plugins/mobile apps. Open-source collaboration promotes progress in the spam detection field.