MixRea: Unveiling the 'Inattentional Blindness' of Large Language Models — A Benchmark for Explicit-Implicit Hybrid Reasoning

Inspired by the theory of inattentional blindness in cognitive psychology, researchers constructed the MixRea benchmark to evaluate the reasoning ability of large language models (LLMs) in scenarios with mixed explicit and implicit information. They found that even state-of-the-art models exhibit attention biases, and proposed the PRCP prompting method to improve reasoning by restoring neglected causal relationships.

In cognitive psychology, 'inattentional blindness' refers to the phenomenon where people ignore obvious but irrelevant stimuli while focusing on a task (e.g., the 'invisible gorilla' experiment). Researchers hypothesize that LLMs, whose training data reflects human attention biases, may have similar 'blindness' phenomena. To this end, they introduced the explicit-implicit hybrid reasoning task, which requires simultaneous processing of directly stated explicit information and implicitly inferred information, making up for the deficiency of traditional benchmarks that focus on a single type of reasoning.

The MixRea benchmark contains 2246 multiple-choice questions, covering 9 reasoning types (causal, counterfactual, multi-hop, commonsense, mathematical, spatial, temporal, social, scientific). For each type, different ratios of explicit and implicit information are designed. The core principle: the correct answer depends on the combination of explicit and implicit information, and incorrect options are set to target common attention biases.

Evaluation of 21 advanced LLMs shows: the state-of-the-art Gemini 2.5 Pro has a consistent accuracy rate of only 42.8%, GPT-4 series range from 38% to 41%, Claude series from 35% to 40%, and open-source models are generally below 35%—far lower than the 70-80% of human experts. Error patterns include over-focus on explicit clues, context neglect, and excessive anchoring to task instructions, which are similar to human cognitive biases.

The Potential Relationship Completion Prompt (PRCP) alleviates inattentional blindness by explicitly completing relationships between concepts. The steps are: concept extraction, relationship completion, constraint identification, and comprehensive reasoning. Experiments show that PRCP improves the consistent reasoning accuracy of models by an average of 8-12 percentage points, with a 15-percentage-point improvement in causal reasoning, and it is also effective for strong models.

In multi-document reasoning, there is over-reliance on main documents; in long-context reasoning, there are position biases and recency effects; in cross-modal reasoning, there are modal preferences and alignment failures. Inattentional blindness is prevalent across multi-source reasoning tasks.

The attention mechanism needs to be rethought (hard attention, multi-scale, explicit relationship modeling); training data requires balanced sampling of explicit and implicit information, adversarial examples, and multi-perspective annotations; the goal of cognitive alignment should pursue complementary intelligence, allowing models to notice details that humans overlook.

Limitations of MixRea: it focuses on English and multiple-choice questions, with limited coverage. Future research needs to explore cross-lingual and cross-cultural performance, underlying mechanisms, and architectural innovations. Conclusion: LLMs have systematic inattentional blindness, which warns against deployment in high-risk scenarios; PRCP brings hope for mitigation, and human-machine collaboration is the future direction.