ImplicitMemBench: A Benchmark for Measuring Unconscious Behavioral Adaptation in Large Language Models

A joint research team from the University of Hong Kong and Harbin Institute of Technology published an Oral paper at ACL 2026, proposing ImplicitMemBench—the first benchmark for systematically evaluating the implicit memory capabilities of large language models (LLMs). Through three types of tasks (procedural memory, classical conditioning, and priming effect), this benchmark explores whether LLMs can acquire, retain, and express unconscious memory through behavioral interactions, providing a new perspective for understanding AI learning mechanisms and safety.

Memory research on large language models has long focused on explicit memory (contextual fact retention), but human cognition involves deeper implicit memory (e.g., unconsciously learned skills like riding a bike or typing). The team raised a core question: Can LLMs acquire, retain, and express implicit memory through behavioral interactions? To address this, they developed the ImplicitMemBench benchmark, filling a gap in this field.

ImplicitMemBench covers three core implicit memory phenomena in cognitive science:

1. Procedural Memory: Tests whether the model unconsciously masters operational patterns through repeated interactions (without explicit instructions);
2. Classical Conditioning: Examines the model's automatic association of repeatedly paired stimulus-response;
3. Priming Effect: Evaluates the implicit guidance of previously encountered content on subsequent behavior (no recall required).

The ImplicitMemBench code repository includes a complete data generation and evaluation framework:

• Data Generation: Supports three types of tasks; parameters like sample count and temperature coefficient can be adjusted via configuration files;
• Evaluation Module: Standardized testing process, supporting batch evaluation via OpenAI-compatible APIs;
• Judgment Mechanism: Procedural memory tasks are rated using sentence-transformers; all tasks use automated judgment to ensure reproducibility.

This research has far-reaching significance:

• Learning Mechanism: If LLMs possess implicit memory, it will change our understanding of AI learning, requiring the redesign of continuous learning and personalized adaptation paradigms;
• Safety Considerations: Implicit memory may lead models to unconsciously form preferences/biases that are difficult to detect or explain, posing AI safety issues.

Researchers can use ImplicitMemBench through the following steps:

1. Configure evaluation/generation files (specify API endpoints, model names, task categories, etc.);
2. Run Python scripts to start experiments;

• The dataset uses the CC BY 4.0 license, the code uses the MIT license, and HuggingFace provides a quick access channel.

ImplicitMemBench marks a shift in LLM research from "what they can do" to "how they learn". Understanding the implicit memory capabilities of LLMs is crucial for building safer and more controllable AI, laying the foundation and pointing the way for future exploration.