DailyClue: A New Benchmark for Visual Reasoning Capabilities of Multimodal Large Models in Daily Scenarios

The Chinese University of Hong Kong, Shanghai AI Lab, and other institutions jointly proposed the DailyClue benchmark, which is specifically designed to evaluate the visual clue-driven reasoning capabilities of multimodal large language models (MLLMs) in daily scenarios. This benchmark covers four major daily domains and 16 sub-tasks, requiring models to actively identify key visual clues and perform reasoning instead of simple object recognition, thus filling the gap in existing evaluations that lack sufficient focus on reasoning capabilities.

Most current evaluation benchmarks for multimodal large language models (MLLMs) focus on assessing the models' prior knowledge or perceptual understanding abilities, but ignore the more critical reasoning capabilities. In daily life, visual scenes are often information-rich and noisy; models need to have the ability to filter key visual clues from complex environments and perform logical reasoning. Existing visual question answering benchmarks usually stay at the level of simple object recognition or surface perception, which cannot truly reflect the models' reasoning performance in complex daily scenarios. This evaluation gap seriously restricts our accurate understanding of the actual capabilities of MLLMs.

The construction of DailyClue follows two core principles:

First, strictly rooted in real daily activities. The research team carefully selected life-like scenarios to ensure that the test data has practical application value, rather than being artificially constructed abstract questions.

Second, challenging query design. The question design goes beyond the surface perception level, requiring models to actively explore appropriate visual clues and perform subsequent reasoning based on these clues, instead of giving answers directly.

DailyClue covers four major daily domains, including home life, outdoor scenes, social interaction, and tool use. Each domain is further divided into 16 different sub-tasks to ensure the comprehensiveness and diversity of the evaluation.

The design of these sub-tasks fully considers the complexity of daily scenarios: models need to identify decisive clues in visually rich environments, filter out irrelevant noise, and perform accurate reasoning based on key information. This "search-reasoning" paradigm is closer to the cognitive process of humans in real life.

The research team conducted a comprehensive evaluation of various mainstream MLLMs and agent models, and the results revealed the severe challenges of this benchmark.

The core finding shows: Accurately identifying visual clues is a necessary prerequisite for robust reasoning. The model's performance in visual clue localization directly determines its reasoning quality. Models that can effectively filter and utilize key visual information show obvious advantages in overall reasoning tasks.

In addition, the evaluation results also exposed several weaknesses of existing models in handling daily scenarios, pointing out the direction for improvement in subsequent research.

The proposal of DailyClue fills an important gap in the evaluation of MLLMs' reasoning capabilities. It not only provides researchers with a standardized testing platform but also, more importantly, redefines the paradigm of multimodal model evaluation—shifting from simple perceptual recognition to deep clue-driven reasoning.

This transformation is crucial for promoting the implementation of multimodal AI in practical applications. Whether it is smart home assistants, autonomous driving systems, or robot interactions, models need to have the ability to perform effective reasoning in complex visual scenarios.

DailyClue opens up a new dimension for the capability evaluation of multimodal large models. With the promotion and application of this benchmark, we look forward to seeing more algorithmic innovations targeting visual clue reasoning, driving MLLMs to make a leap from "understanding what they see" to "thinking through what they see".