MECO: A Multimodal Emotion and Cognition Understanding Dataset for Elderly Populations

MECO is the first multimodal emotion and cognition understanding dataset specifically for elderly populations, including 42 participants, 38 hours of multimodal signals (video, audio, EEG, ECG), and 30,592 synchronized samples. This dataset fills the data gap in affective computing and cognition research for the elderly, providing an important foundational resource for applications such as early detection of mild cognitive impairment (MCI).

Although the field of affective computing has made progress, existing multimodal emotion datasets almost exclusively focus on young and healthy populations, neglecting the elderly (especially those with cognitive decline). Global aging is accelerating, and the incidence of cognitive impairment is rising. Emotional abnormalities are early signals of cognitive decline, so developing emotion recognition technology for the elderly is crucial for early MCI detection.

The MECO dataset includes 42 elderly participants, 38 hours of multimodal signals, and 30,592 synchronized samples, with a focus on ecological validity (collected in community settings). It covers four modalities: video (facial expressions, etc.), audio (acoustic features), EEG (neurophysiological indicators), and ECG (autonomic nervous system indicators). Annotations include emotion (valence, arousal, six basic emotions) and cognitive status (MMSE scores).

The research team established baseline benchmarks for emotion and cognition prediction, using mainstream multimodal fusion methods. Results show that multimodal fusion outperforms single modalities, verifying the dataset's quality. It also reveals challenges in emotion recognition for the elderly (e.g., reduced amplitude of facial expressions), pointing the way for future research (robust feature extraction, heterogeneous modality fusion, etc.).

Applications of MECO include: 1. Personalized emotion recognition (adapting to individual differences among the elderly); 2. Early MCI detection (analyzing the relationship between emotional patterns and MMSE); 3. Emotional support systems (real-time emotion recognition and response); 4. Aging research (exploring the relationship between emotion and cognition).

The MECO dataset is openly accessible via the project website (https://maitrechen.github.io/meco-page/), adhering to open science principles and ethical guidelines, with anonymization to protect privacy. The community is encouraged to use it and provide feedback, and the team will continue to maintain and update it.

MECO has limitations: the sample size needs to be expanded, cultural background is single, there is a lack of longitudinal tracking, and signal types can be extended. Future plans: increase sample size and diversity, conduct cross-cultural collection, carry out longitudinal studies, and integrate more physiological signals.

MECO marks a step towards aging-friendly affective computing, recognizing the elderly as an important user group. It lays the foundation for developing AI systems that understand the emotional needs of the elderly, improving quality of life and supporting healthy aging. It emphasizes that technology should serve all groups, especially those historically neglected.