Deepfake Audio Detection System Based on PyTorch and MFCC Features

This project is an open-source deepfake audio detection system released by SoumilPatria on GitHub. Its core technology combines MFCC feature extraction with a CNN classifier (based on the PyTorch framework), enabling high-precision recognition of AI-generated audio. The test set accuracy reaches 97.67% with an equal error rate of 1.93%. It also provides a Streamlit web application for non-technical users, aiming to address the information security challenges posed by deepfake audio.

With the development of generative AI technology, the risk of deepfake audio abuse is on the rise (e.g., scam calls, voiceovers for fake news). Traditional audio analysis methods struggle to handle the complexity of modern AI-synthesized speech, so a dedicated deep learning detection solution is needed.

The project uses a standardized subset of the Fake-or-Real dataset (2-second clips), which includes real and AI-generated speech samples. It employs MFCC feature extraction technology, converting raw audio into 2D feature maps via the librosa library to simulate the non-linear frequency perception characteristics of the human auditory system.

A custom CNN classifier is built based on PyTorch, suitable for processing the 2D spatial structure of MFCC feature maps. Local patterns are extracted via convolutional layers, dimensions are reduced via pooling layers, and finally, a binary classification result (real/fake) is output.

Core metrics: Overall accuracy 97.67%, equal error rate 1.93%, real speech recognition rate 96.14%, fake speech recognition rate 99.21%. The confusion matrix shows only 11 fake samples were misclassified as real, and the conservative detection strategy is suitable for practical applications.

A web application interface built with Streamlit is provided, allowing users to upload audio files and get detection results in real time. The end-to-end solution lowers the usage threshold and is suitable for researchers, content review teams, media organizations, and security departments.

1. Combining classic MFCC features with CNN deep learning is more effective than end-to-end learning; 2. The lightweight solution is suitable for resource-constrained environments; 3. The complete chain from training scripts to web applications reflects a practical-oriented design.

The project has reached a practical level, providing an effective solution to address the challenges of AI-generated content. In the future, it can integrate more generative model samples, explore attention mechanisms, and develop real-time detection capabilities, serving as a basic reference for subsequent research.