GRU-based Neural Network Auditory Attention Decoding System: From EEG Signals to Real-Time Neural Interfaces

This project proposes a complete Auditory Attention Decoding (AAD) pipeline using a Gated Recurrent Unit (GRU) deep learning architecture. By analyzing EEG signals, it determines which speaker the listener is focusing on, achieving an accuracy of 85.6% with a 0.25-second decision window, providing a technical foundation for applications like neuro-controlled hearing aids.

Project Background and Core Challenges

Auditory Attention Decoding (AAD) is an important research direction in the field of neural engineering. Its core goal is: when a listener is in an environment where multiple people are speaking simultaneously, the system can determine which speaker he/she is focusing on by analyzing their EEG activity. This technology is of great significance for the development of "neuro-controlled hearing aids"—future smart hearing aids can automatically amplify the sound of the target speaker according to the user's attention direction while suppressing other interfering sound sources.

Traditional AAD methods usually rely on long decision windows (on the order of seconds) to obtain stable decoding results, but this introduces significant delays, making it difficult to meet the needs of real-time applications. How to shorten the decision time while maintaining high accuracy is a key technical challenge in this field.

Technical Architecture and Implementation Plan

This project adopts a deep learning architecture based on Gated Recurrent Units (GRU), specifically designed to process time-series EEG signals and speech envelope data. The entire system consists of three parallel GRU streams:

• EEG Signal Stream: Processes 64-channel EEG signals to capture the brain's neural responses to auditory stimuli
• Speaker 1 Speech Envelope Stream: Analyzes the speech features of the left speaker
• Speaker 2 Speech Envelope Stream: Analyzes the speech features of the right speaker

Before the EEG stream enters the GRU, the system first weights each EEG channel through a Channel-Attention Module to highlight key channel information related to auditory attention. The final hidden states of the three streams are compared and processed through a fully connected layer to output a binary classification result: whether the listener is focusing on the left or right speaker.

Data Preprocessing and Experimental Design

The project uses the public auditory attention detection dataset from KU Leuven, which includes EEG records of 16 subjects and corresponding audio stimuli. Data preprocessing is divided into two main stages:

MATLAB Preprocessing Stage:

• Load raw EEG data (.mat format) and audio stimulus files
• Apply filtering and downsampling to process EEG signals
• Use gammatone filters to extract speech envelopes from audio
• Synchronize the timestamps of EEG signals and audio signals

Python Segmentation Stage:

• Split continuous data into fixed-length decision windows
• Test four window lengths: 0.25s, 0.5s, 1.0s, 2.0s
• Generate training samples containing EEG windows, speech envelopes of two speakers, and attention labels

Model Training and Data Augmentation Strategies

Training uses a subject-specific design, i.e., a separate model is trained for each subject. This design considers individual differences in EEG signals and can achieve more stable decoding performance.

To improve the model's robustness and generalization ability, the project implements four data augmentation methods:

1. Gaussian Noise Injection: Add random noise to EEG signals to simulate signal interference in real acquisition
2. EEG Channel Random Dropout: Randomly mask some EEG channels to enhance the model's adaptability to missing data
3. Speech Envelope Amplitude Scaling: Adjust the amplitude of speech signals to simulate speaking scenarios with different volumes
4. Time Warping: Stretch or compress the speech envelope in time to enhance the model's robustness to speech rate changes

Experimental Results and Performance Analysis

The experimental results reveal a counterintuitive finding: the shortest decision window achieved the highest decoding accuracy. The specific results are as follows:

Decision Window Length	Average Test Accuracy
0.25s	~85.6%
0.5s	~84.6%
1.0s	~81.6%
2.0s	~73.1%

This result indicates that the GRU model can effectively capture short-term dynamic features in EEG signals, supporting near-real-time auditory attention decoding. Shorter windows may reduce the accumulation of non-stationary noise, and the GRU's gating mechanism can selectively retain key temporal information related to attention.

It is worth noting that the data augmentation strategy did not outperform the baseline model in all cases, but it provided valuable insights into understanding model robustness and the impact of input perturbations.

Technical Transferability and Application Prospects

Although this project focuses on the field of neural technology, its engineering implementation mode has broad transfer value:

• Multimodal Data Preprocessing: The synchronous processing flow of EEG and audio signals can be extended to other multimodal scenarios
• Time Series Segmentation Strategy: The design of sliding windows and overlapping sampling is applicable to various time-series prediction tasks
• Attention Mechanism Design: The channel attention module can be transferred to other multi-channel sensor data processing
• Deep Learning Experiment Framework: The complete flow from data loading, model training to result analysis is universal

At the application level, this technology provides a feasible technical path for neuro-controlled hearing aids, brain-computer interface (BCI) systems, and cognitive state monitoring devices. Future improvement directions include cross-subject generalization, stricter training/test separation strategies, and systematic hyperparameter optimization.