# SMMU: A Benchmark Framework for Social Intelligence of Multimodal Large Language Models > SMMU is a benchmark project specifically designed to evaluate the social intelligence capabilities of multimodal large language models, testing their Theory of Mind, social reasoning, and situational understanding through interactive scenarios. - 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm) - 发布时间: 2026-05-24T06:29:34.000Z - 最近活动: 2026-05-24T06:53:22.483Z - 热度: 161.6 - 关键词: SMMU, 社交智能, 基准测试, 多模态大语言模型, Theory of Mind, 心理理论, 情绪理解, AI评估, GitHub - 页面链接: https://www.zingnex.cn/en/forum/thread/smmu-7464fa90 - Canonical: https://www.zingnex.cn/forum/thread/smmu-7464fa90 - Markdown 来源: floors_fallback --- ## [Introduction] SMMU: A Benchmark Framework for Social Intelligence of Multimodal Large Language Models SMMU is a benchmark project dedicated to evaluating the social intelligence capabilities of multimodal large language models, filling the gap in current LLM benchmarks that focus on cognitive abilities but lack systematic assessment of social intelligence. It tests models' core social intelligence capabilities such as Theory of Mind, social reasoning, and situational understanding through interactive scenarios. Adopting an open-source model to support community contributions, it provides a standardized evaluation tool for the development of social intelligence in multimodal models. ## Project Background and Motivation Current large language model (LLM) benchmarks mostly focus on cognitive abilities such as language understanding, mathematical reasoning, or code generation, but lack systematic evaluation of models' performance in social contexts—namely, 'social intelligence'. Social intelligence is the core of human intelligence, encompassing the ability to understand others' intentions, emotions, beliefs, and to respond appropriately in social scenarios. The SMMU project aims to fill this gap and provide a comprehensive social intelligence evaluation framework for multimodal large language models. ## What is Social Intelligence? Social intelligence refers to an individual's ability to understand and manage social relationships, infer others' mental states, and respond appropriately in social contexts. Its key components include: - **Theory of Mind**: Understanding that others have beliefs, desires, and intentions different from one's own - **Emotion Recognition**: Identifying emotional states from language, expressions, or contexts - **Social Reasoning**: Predicting behaviors or outcomes based on social cues - **Situational Awareness**: Understanding social norms and context-dependent behaviors For multimodal models, social intelligence also requires integrating visual information (e.g., facial expressions, body language) with textual information to form a complete understanding of social scenarios. ## Core Design: Evaluation Dimensions and Scenario Principles ### Evaluation Dimensions SMMU evaluates models' social intelligence from the following dimensions: 1. Belief Inference: Testing the ability to understand others' false beliefs 2. Emotion Understanding: Assessing the ability to identify and explain emotional states 3. Intention Recognition: Inferring others' intentions from behaviors or dialogues 4. Social Norm Compliance: Testing understanding of social rules and etiquette 5. Multimodal Integration: Conducting social reasoning by combining visual and textual information ### Scenario Design Principles - Naturalness: Scenarios are derived from real social interaction contexts - Progressive Difficulty: From simple emotion recognition to complex multi-turn social reasoning - Multimodal Fusion: Combining image, video, and textual information - Cultural Neutrality: Avoiding biases from specific cultural backgrounds SMMU adopts an interactive evaluation method, which is closer to real social interactions and accurately assesses models' social reasoning abilities. ## Technical Implementation and Architecture The SMMU project is built using JavaScript and includes an interactive web interface (deployed via GitHub Pages). The project structure is as follows: - **Frontend Interface**: Provides intuitive display of test scenarios and interaction - **Evaluation Logic**: Built-in scoring mechanism to automatically evaluate model responses - **Dataset**: Predefined social scenarios and expected response standards Online access to the project: https://smmu-team.github.io/SMMU/, researchers can directly experience the benchmark test in their browsers. ## Importance of Social Intelligence Evaluation ### Practical Application Scenarios Social intelligence evaluation is crucial for the following applications: - Dialogue Assistants: Understanding users' emotions and providing empathetic responses - Educational Tutoring: Adapting to learners' emotional and cognitive states - Mental Health Support: Identifying signs of emotional distress in users - Customer Service: Understanding customers' emotions and intentions, and providing appropriate responses - Social Robots: Engaging in natural social interactions with humans ### Model Development Directions Through the SMMU benchmark, researchers can: - Identify the shortcomings of current models in social reasoning - Guide model training to targetedly improve social intelligence - Compare the social ability performance of different models - Track the progress of models' social intelligence ## Usage and Contribution: Open-Source Community Participation SMMU adopts an open-source model and welcomes community contributions: - Use the existing benchmark to test your own models - Submit new social scenarios to expand the test set - Improve evaluation metrics and scoring mechanisms - Share test results and findings Original author/maintainer: SMMU-Team, Source platform: GitHub, Original link: https://github.com/SMMU-Team/SMMU, Release date: May 24, 2026. ## Summary and Outlook SMMU represents an important direction in the AI evaluation field, expanding from pure cognitive ability testing to social intelligence evaluation. As AI systems increasingly participate in human social interactions, evaluating and improving their social understanding capabilities is crucial. This benchmark not only provides researchers with a standardized evaluation tool but also points the way for the future development of social intelligence in multimodal models. We look forward to more models making progress on this benchmark, ultimately achieving AI systems with true social understanding capabilities.