Reasoning as Pattern Matching: The Shared Daily Reasoning Mechanism Between Humans and Large Models

A recent arXiv study challenges the traditional view that 'humans reason using abstract world models while large language models (LLMs) only perform pattern matching', finding that humans and LLMs have highly similar error patterns in daily reasoning. The study reveals the pattern matching mechanism of LLMs through attention head analysis and can predict human reasoning errors influenced by irrelevant prompt details, providing a new perspective for understanding the nature of intelligent reasoning.

When LLMs fail at reasoning tasks, a common explanation is 'they don't truly reason, just pattern match', implying the assumption that human reasoning is based on abstract world models and does not make similar errors. This binary opposition view of 'humans = true reasoning, LLMs = pattern matching' is widely prevalent in the AI field, but the study questions: Is human daily reasoning really perfect without such errors?

1. Comparative Experiments: Design large-scale experiments to evaluate the common sense reasoning ability of human participants and 25 LLMs in daily causal reasoning scenarios, focusing on natural reasoning processes rather than formal logical problems;
2. Attention Head Analysis: Identify key attention heads that affect reasoning outputs through intervention experiments and activation analysis, and explore their underlying mechanisms.

1. Similar Error Patterns: Humans and LLMs make systematically similar errors on specific types of reasoning problems, and certain reasoning tasks are equally difficult for both;
2. Cross-Species Prediction: Analyzing key attention heads of LLMs can predict human reasoning errors influenced by irrelevant prompt details (e.g., misleading surface features like wording and vocabulary), known as the 'irrelevant prompt detail effect'.

The study challenges the traditional binary opposition and proposes that human daily reasoning is closer to complex pattern matching rather than pure formal logical operations. This echoes psychological research on 'heuristics': both rely on statistical patterns extracted from experience rather than strict logical rules, prompting a rethinking of the nature of reasoning.

For AI System Design:

1. Should not overly criticize LLM errors (they reflect human systematic biases);
2. Can draw on human cognitive bias research to improve LLMs;
3. Emphasize prompt engineering to avoid misleading patterns.

For Cognitive Science: LLMs can serve as computational models for studying human reasoning, and new insights into human cognition can be gained by analyzing their attention mechanisms.

Limitations: Only focuses on daily common sense reasoning, not covering fields like formal logic and mathematical proof; the definition of 'pattern matching' needs to be more precise.

Future Directions:

1. Cross-domain validation (mathematics, science, moral reasoning, etc.);
2. Compare LLM attention mechanisms with human brain neural activities;
3. Develop technologies to reduce systematic reasoning biases (for AI systems and human education).