# SMMU Benchmark: Evaluating Social Intelligence of Multimodal Large Language Models

> Introduces the SMMU project, a benchmark framework specifically designed to evaluate the social intelligence capabilities of multimodal large language models, filling a critical gap in the current AI evaluation system.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-22T06:03:20.000Z
- 最近活动: 2026-05-22T06:20:31.226Z
- 热度: 141.7
- 关键词: 多模态大模型, 社交智能, 基准测试, 心智理论, 情绪识别, AI评估, 人机交互, 认知智能
- 页面链接: https://www.zingnex.cn/en/forum/thread/smmu-e6c42232
- Canonical: https://www.zingnex.cn/forum/thread/smmu-e6c42232
- Markdown 来源: floors_fallback

---

## [Introduction] SMMU Benchmark: Filling the Gap in Social Intelligence Evaluation for Multimodal Large Models

This article introduces the SMMU (Social Intelligence Benchmark for Multimodal Understanding) project, a benchmark framework specifically designed to evaluate the social intelligence capabilities of multimodal large language models. While the current AI evaluation system is comprehensive, it has long overlooked social intelligence—a core capability that is crucial for AI to integrate into human society. By deconstructing social intelligence into multiple dimensions (emotion recognition, theory of mind, social context understanding, reasoning and prediction), and adopting a multimodal test design and hybrid evaluation method, SMMU fills this evaluation gap, providing model developers and researchers with diagnostic tools and a common platform to drive AI evaluation toward a direction that is closer to human real-world capabilities.

## Background: Social Intelligence is a Long-overlooked Core Capability in AI Evaluation

The current evaluation system for multimodal large language models covers multiple dimensions such as visual question answering and image captioning, but social intelligence—a core component of human intelligence—has been long overlooked. Social intelligence involves understanding others' emotions, intentions, and beliefs, interpreting social context clues, and predicting the trajectory of interpersonal interactions. It is crucial for AI systems that serve humans; the practical value of AI lacking social intelligence will be greatly reduced.

## Overview of the SMMU Project: A Multidimensional Evaluation Framework for Social Intelligence

The SMMU project aims to fill the gap in social intelligence evaluation, with its core contribution being the definition of the problem space for social intelligence evaluation and the construction of a standardized dataset. The framework decomposes social intelligence into four sub-dimensions: basic-level emotion recognition (identifying non-verbal cues such as facial expressions and body language); deep-level theory of mind (understanding others' different beliefs and intentions); social context understanding (grasping scene norms and cultural backgrounds); and the highest-level social reasoning and prediction (predicting the development of situations and behavioral consequences).

## Methodology: Multimodal Fusion Test Design and Hybrid Evaluation Metrics

A notable feature of SMMU is its multimodal design; test materials integrate visual (images, videos) and linguistic information, closely resembling real social scenarios. The evaluation uses a hybrid model of multiple-choice questions and open-ended answers: multiple-choice questions facilitate large-scale automated evaluation, while open-ended answers reveal details of reasoning. In addition to accuracy, metrics also focus on confidence calibration, hierarchical performance, and adversarial robustness to comprehensively reflect the behavioral characteristics of models.

## Evidence: Limitations of Existing Models in Social Intelligence

Preliminary evaluations based on SMMU show that existing models perform well in basic emotion recognition tasks, but there is a significant gap between them and humans in complex theory of mind and social reasoning tasks. Failure mode analysis found that models are prone to systematic errors when integrating multiple social cues, handling long-term relationship dynamics, or dealing with culture-specific norms, which triggers reflections on training data and pre-training objectives.

## Conclusions and Recommendations: Application Value and Future Directions of SMMU

The application value of SMMU includes: providing developers with diagnostic tools to help identify shortcomings in social intelligence; and establishing a fair comparison platform for researchers. Future directions can include expanding the types of social scenarios, adding cross-cultural samples, developing fine-grained error analysis tools, and exploring the relationship between social intelligence and other cognitive abilities. Social intelligence is the cornerstone of human-AI collaboration; its evaluation and cultivation should become an important part of AI development to facilitate AI safety and alignment.