Music Genre Classification Using CNN and Transfer Learning: A GTZAN Dataset Practice

This article introduces the statistical learning course project of Beatrice Malvezzi, a student at the University of Milan-Bicocca, which explores the application of CNN and transfer learning to music genre classification. The core idea is to convert audio into Mel spectrograms, use CNN to extract features, compare methods such as custom CNN and VGG16 transfer learning, and finally achieve an accuracy of over 94% on the GTZAN dataset. The project also discusses overfitting suppression techniques, providing a reference for audio classification.

Automatic music genre classification is an important direction in audio processing and machine learning. Traditional methods rely on manual feature engineering, while this project attempts to transfer CNN technology from computer vision to audio classification: converting audio into Mel spectrograms (visual representations), using CNN's image feature extraction capabilities to identify genre patterns, and avoiding complex manual feature design. This project is an individual project for the statistical learning course at the University of Milan-Bicocca.

The classic GTZAN dataset is used (1000 audio files, 10 genres: Blues, Classical, Country, Disco, Hip-hop, Jazz, Metal, Pop, Reggae, Rock). Each audio is converted into a 224×224 RGB Mel spectrogram to retain information about frequency changes over time. The dataset is divided into a training set (918 images, 70%), validation set (273 images, 15%), and test set (280 images, 15%).

Compare multiple models:

1. Basic CNN: 4 convolutional layers + pooling + Dropout(0.4) fully connected layer, Adam optimizer, test accuracy of 92.5% but with overfitting.
2. Tuned CNN: Grid search optimized Dropout(0.4) and learning rate(0.0001), accuracy increased to 93.57%, loss reduced to 0.205.
3. VGG16 Transfer Learning: Freeze the pre-trained convolutional base, add a custom classifier (including BatchNorm, Dropout, L2 regularization), validation accuracy of approximately 94.87%.
4. VGG16 Fine-tuning: Unfreeze layers from block3_conv1 onwards, train with a low learning rate, validation accuracy stabilized around 94%.
5. Feedforward Neural Network: Only fully connected layers, accuracy of only 10.36%, proving the key role of CNN in capturing spatial features.

Model	Test Accuracy	Test Loss
Initial CNN	92.50%	0.384
Tuned CNN	93.57%	0.205
VGG16 Feature Extraction	~94-95%	-
Feedforward Neural Network	10.36%	2.302
Conclusion: Transfer learning + pre-trained models significantly improve performance; a pure fully connected network cannot effectively learn features.

The project is implemented in R language, relying on the keras and tensorflow packages. The code is organized in RMarkdown for easy reproduction. Regularization techniques include: Dropout (prevents co-adaptation), Batch Normalization (stabilizes training), L2 regularization (restricts weights), Early Stopping (prevents overfitting), and ReduceLROnPlateau (adaptively adjusts learning rate).

Improvement Directions: Combine MFCC and other original audio features with spectrograms; use RNN/LSTM to capture time dependencies; data augmentation for spectrograms; try models like ResNet/EfficientNet. Practical Significance: Demonstrate the feasibility of cross-domain technology transfer (CV→audio); provide a case for beginners in deep learning audio processing; emphasize the importance of overfitting suppression (the decrease in test loss reflects improved generalization ability).