Automatic News Classification and Hot Topic Detection: A Multi-Model Fusion Practice

This article introduces a machine learning-based automatic news classification and hot topic detection system. By combining TF-IDF, sentence embeddings, and multiple classification models (Multi-Layer Perceptron, Logistic Regression, XGBoost), it achieves an 87% classification accuracy. The system also uses clustering and time series analysis to detect hot topics, making it a lightweight and deployable solution.

In the era of information explosion, the number of news articles is growing exponentially. Manual classification is costly and inefficient. Automatic news classification has become an important direction in natural language processing. This open-source project provides an end-to-end solution through a combination of classic machine learning methods.

The system uses two complementary features:

1. TF-IDF Vectorization: Captures the importance of words, has strong interpretability, and is suitable for vocabulary recognition in the news domain;
2. Sentence Embedding: Pre-trained models encode semantic vectors to identify texts with similar meanings, complementing TF-IDF.

The project compares three classification algorithms:

• MLP: Learns non-linear combinations of features, balancing capacity and efficiency;
• Logistic Regression: A baseline model with fast training and good interpretability;
• XGBoost: An ensemble of decision trees that captures high-order feature interactions and performs best in most categories.

Hot topic detection clusters similar news into topic groups and tracks growth trends over time to identify hotspots. It does not require predefined topics, adaptively discovers emerging hotspots, and provides representative keywords.

Achieved an 87% accuracy rate on real datasets. Key findings:

• Feature fusion outperforms single features;
• XGBoost performs best, with a small gap from Logistic Regression;
• Technology categories are easy to distinguish, and titles are more discriminative than the main text.

Value: Lightweight, can be deployed on ordinary servers, suitable for resource-constrained scenarios; Limitations: Requires manual review of misclassified items, sensitive to data distribution, and lacks multilingual support.

The project demonstrates a practical solution approach, with technology selection serving the task itself. Future recommendations: Optimize accuracy, support multilingualism, and enhance data adaptability.