Acoustic-ESP: Acoustic Radar Localization Using Stereo Audio and Machine Learning

Project Basic Info

• Author/Maintainer: Himel54
• Source: GitHub (https://github.com/Himel54/acoustic-esp)
• Release Time: 2026-05-27

Core Overview

Acoustic-ESP is an open-source acoustic radar project using ESP32 microcontroller, stereo audio input, and machine learning to estimate sound source direction and distance. It provides a low-cost solution for scenarios like game interaction, robot navigation, and smart home.

The following floors will cover technical background, application scenarios, implementation details, significance, limitations, and conclusion.

Traditional Challenges

Acoustic localization isn't new, but real-time, low-power implementation on microcontrollers was challenging—traditional methods needed complex arrays or expensive sensors.

Project's Core Approach

1. Stereo Audio Collection: Uses dual microphones to capture signals, inferring direction via Time Difference of Arrival (TDOA) and intensity difference.
2. Machine Learning Model: Unlike physics-based methods, ML offers better environment adaptability (reduces echo/noise), nonlinear compensation, and generalization.
3. ESP32 Advantages: Low cost, integrated Wi-Fi/Bluetooth, sufficient computing power (dual-core for audio collection and light inference), low power for portable devices.

Game Interaction

• VR/AR games: Track player position or sound event direction without light dependency, better privacy than camera-based tracking.

Robot Navigation

• Detect obstacles or locate sound sources (e.g., human calls for help, alarms) in visually limited environments (smoke, darkness).

Smart Home

• Intrusion Detection: Locate abnormal sounds (glass breaking).
• Baby Monitoring: Track baby cry positions.
• Voice Assistant Enhancement: Accurately judge user's speaking direction.

Audio Preprocessing

1. Sampling & Filtering: 16kHz+ sampling rate with band-pass filtering to remove irrelevant frequencies.
2. Framing & Windowing: Split audio into short frames with Hamming window.
3. Feature Extraction: Extract MFCC, spectrogram, or other features suitable for neural networks.

Model Architecture (Inferred from Similar Projects)

Possible models: Convolutional Neural Networks (CNN) for 2D features like spectrograms; RNN/LSTM for time dynamics; fully connected networks as regression heads for direction/distance output.

Dataset & Training

Requires labeled data: sound samples from different directions/distances, diverse environments (reverb, noise), various sound types (human voice, music, ambient sounds).

Acoustic-ESP democratizes complex acoustic localization, making it runnable on cheap microcontrollers.

For developers:

• Learning Resource: Understand how to apply ML to embedded audio processing.
• Extensible Framework: Serves as a base for more complex acoustic applications.
• Innovation Inspiration: Shows multiple possibilities of acoustic sensing.

Current Limitations

• Precision: Lower than professional acoustic arrays due to dual-mic setup.
• Environment Dependency: Performance affected by differences between training and deployment environments.
• Sound Type: May perform poorly on certain frequencies/types of sounds.

Potential Improvements

• Multi-Mic Array: Increase microphone count to boost precision.
• Adaptive Algorithms: Implement online learning or domain adaptation for better environment adaptability.
• Multi-Modal Fusion: Combine with visual or inertial sensor data.

Acoustic-ESP cleverly combines ML, embedded systems, and acoustic engineering. It demonstrates how to implement practical intelligent functions on resource-constrained devices, providing valuable references for IoT and edge AI applications. As embedded ML technology advances, more such innovative projects are expected.