Hand-drawn Image Recognition: CNN Classifier for Hand-drawn Objects Based on the TU-Berlin Dataset

This project trains a Convolutional Neural Network (CNN) classifier based on the TU-Berlin hand-drawn dataset to achieve real-time recognition and interactive demonstration of hand-drawn objects. The core content includes dataset characteristics, model optimization design, performance, application scenarios, and future directions, which will be analyzed in detail in the following floors.

The TU-Berlin hand-drawn dataset is a highly influential public dataset in the field: it contains over 20,000 sketches covering 250 categories of daily objects, drawn by non-professional artists within a time limit, and stored in vector format that can be rendered at any resolution. Compared to standard image datasets (e.g., ImageNet), hand-drawn images have characteristics such as black-and-white/grayscale, no texture, blank background, conceptual perspective, and large style variations, making it difficult to directly transfer pre-trained models.

Model Architecture: An optimized CNN is used, including feature extraction layers (convolution, pooling, batch normalization, Dropout) and a classification layer (fully connected + Softmax to output probabilities for 250 categories). Hand-drawn Specific Optimization: Data augmentation (random rotation, scaling, elastic deformation, Gaussian noise), input preprocessing (inversion, normalization to [-1,1], uniform size), and class balance processing. Training Strategy: 80% training /10% validation /10% test split, Adam optimizer, cosine annealing/step decay learning rate, early stopping mechanism, and model ensemble to improve robustness.

Performance: On the TU-Berlin test set, the Top-1 accuracy is 55-65%, Top-5 accuracy is 80-85%, and the inference speed is <100ms on GPU and <500ms on CPU. Interactive Demonstration: Supports canvas drawing, real-time prediction, Top-K candidate display, and confidence visualization; the tech stack includes front-end HTML5 Canvas/React, back-end Flask/FastAPI, and model deployment with ONNX/TensorFlow.js.

Education Field: Children's drawing recognition, art teaching evaluation, conceptual design transformation; Creative Tools: Icon search, prototype design element recognition, game interaction control; Assistive Technology: Handwritten formula recognition, gesture command recognition, shorthand sketch to text description.

Challenge 1: Large Style Variations → Strong data augmentation to simulate diverse styles, large models to learn abstract features, style normalization preprocessing; Challenge 2: High Abstraction → Attention mechanism to focus on key areas, multi-scale feature fusion, knowledge distillation to transfer knowledge from photo models; Challenge3: Inter-class Similarity → Fine-grained feature learning, hard example mining, hierarchical classification (coarse categories first, then fine categories).

This project demonstrates the application of CNN in non-traditional visual tasks, testing the model's ability to express abstract concepts. Future directions include multi-modal fusion (combining text descriptions to assist recognition), sequence modeling (predicting while drawing), cross-domain transfer (few-shot/continuous learning to adapt to new categories), etc.