Spatial LDA: An Unsupervised Image Clustering and Topic Modeling Method Combining SIFT and CNN

Introducing the spatial_LDA project, an unsupervised image clustering framework that combines the traditional SIFT algorithm with deep learning CNN features, using the LDA topic model to enable automatic image grouping and annotation assistance. The project is maintained by Ryan Sander, Crystal Wang, and Yaateh Richardson, sourced from GitHub, with the related paper "Unsupervised Image Clustering and Topic Modeling for Accelerated Annotation" published on 2026-05-30.

In the field of computer vision, the performance of supervised learning models highly depends on large-scale annotated data. However, manual annotation requires drawing bounding boxes, segmentation masks, or category labels for each image, which is time-consuming and labor-intensive, becoming a major bottleneck in practical applications. spatial_LDA proposes an unsupervised learning solution that automatically discovers the latent feature structure of images, groups unannotated images into topic categories, and significantly accelerates the data annotation process.

The project architecture consists of four stages:

1. Local Feature Extraction: Uses the SIFT algorithm to extract up to 300 key points, each as a 128-dimensional vector with scale and rotation invariance;
2. Global Semantic Features: Uses an ImageNet pre-trained CNN to extract activation values from the second-to-last and third-to-last layers as global features;
3. Feature Discretization: Merges SIFT and CNN features via K-Means clustering to generate 300 visual words, forming a Visual Bag of Words (VBOW);
4. Topic Modeling: Applies the LDA model to model the image set into 20 latent topics, enabling automatic image grouping.

The project was evaluated on the ADE20K dataset (containing 150 categories of indoor and outdoor scene images with semantic segmentation annotations):

• Symmetric KL divergence was used to evaluate LDA topic quality, and L2 norm was used to evaluate K-Means performance;
• The comparison benchmarks were PCA (classical dimensionality reduction) and VAE (generative model);
• Results show that spatial_LDA can effectively group semantically similar images, with optimal hyperparameters being 300 clustering centers, 300 key points per image, and 20 LDA topics.

The applications of the spatial_LDA framework include:

1. Annotation Acceleration: Batch processing similar images by topic to improve annotation efficiency;
2. Data Curation: Quickly discovering the latent structure of large-scale unannotated image libraries;
3. Active Learning: Using topic model uncertainty to sample information-rich samples;
4. Cross-Domain Transfer: Pre-trained CNN features support cross-domain generalization, allowing application to new image domains without retraining.

The project provides a complete Python implementation with core files:

• lda.py: Main pipeline script supporting the full SIFT-CNN-KMeans-LDA workflow;
• feature_extraction.py: SIFT feature extraction and K-Means clustering;
• dataset.py: Dataset loading and preprocessing;
• eval_k_means_call.py: Evaluation framework;
• pca.py/vae.py: Implementation of benchmark methods. Dependencies can be installed via conda or pip to reproduce experiments, and there are papers and poster documents explaining the theoretical basis and details.

spatial_LDA combines the local feature capability of traditional SIFT, the semantic understanding capability of CNN, and the topic modeling capability of LDA, providing a powerful and flexible framework for unsupervised image analysis and alleviating the annotation bottleneck of supervised learning. In the future, it can be extended to complex scenarios such as video analysis and multi-modal data fusion, promoting the automation and intelligentization process in the field of computer vision.