Using AirPods to Recognize Food via Chewing: A New Approach to Dietary Monitoring on Wearable Devices

Core Idea: An innovative study explores using the built-in IMU sensors in AirPods Pro to capture chewing movements and uses machine learning models to identify different food types. It provides a privacy-friendly and easy-to-stick-to new technical solution for passive dietary tracking, addressing the limitations of traditional dietary tracking methods (tedious manual recording, privacy concerns with camera-based solutions).

Dietary tracking is an important part of health management, but existing solutions have obvious limitations: traditional manual recording requires continuous time and effort, and it's easy to give up due to tediousness; camera-based food recognition has high automation but involves privacy concerns and requires active shooting. A German research project proposes using daily-worn AirPods Pro to capture chewing signals for food recognition, seeking a more passive and privacy-friendly solution.

AirPods Pro have a built-in high-precision Inertial Measurement Unit (IMU, including accelerometer and gyroscope), originally used for spatial audio and noise reduction, which collects motion state data at a frequency of about 50Hz. When chewing, jaw movements are transmitted to the ear canal through bone conduction and mechanical vibration, and captured by the sensors. Differences in physical properties of different foods (such as crisp and hard apples, elastic chewing gum, liquid yogurt) form unique "sound fingerprints" in chewing frequency, amplitude, and rhythm, providing distinguishable features for machine learning classification.

The research team used the Sensor Logger iOS app to extract raw sensor data (acceleration, angular velocity, device orientation) from AirPods Pro. The experiment set four categories: apple, chewing gum, Icelandic yogurt (skyr), and static state (non-eating). 12 rounds of data collection sessions have been completed, each recording the sensor data of subjects when eating specific foods, providing clearly labeled samples for model training.

The research team extracted 37 features from the raw signals (time-domain statistics such as mean/variance/peak value, frequency-domain features, time-series pattern descriptors) and input them into a random forest classifier for training. The preliminary experiment achieved a 92% classification accuracy on 12 session datasets under the leave-one-out cross-validation strategy, indicating that the chewing signals captured by AirPods contain sufficient information about food types.

The current project is in the early stage, and the 92% accuracy is based on a limited dataset; the model's generalization ability needs to be verified. The subsequent key work includes feature selection optimization, larger-scale data collection, and more robust model evaluation. If the technology matures, future applications can include automatic dietary recording, identification of unhealthy snacking habits, and monitoring of chewing disorders.

The ChewML project demonstrates the new potential of wearable devices: smart headphones are not only audio devices but also human sensor nodes. The development of edge computing and TinyML technology makes it possible to run lightweight inference models locally on headphones without uploading sensitive data to the cloud. This "invisible computing" paradigm (technology integrated into daily items, providing services unconsciously) may be the ultimate form of wearable device development.