Claude Code Cognitive Annotation Plugin: Analyzing Cognitive Behavior Evidence in Dialogues Using a Multi-Agent Architecture

This article introduces a Claude Code-based cognitive annotation plugin that uses four agents to analyze dialogue records in parallel, extracting cognitive behavior evidence across four dimensions—executive functions, metacognition, memory and reasoning, and user mental models—to provide structured data support for AGI research. The plugin is zero-dependency, purely native to Claude Code, and follows scientifically rigorous annotation principles, helping to understand the deep cognitive processes in human-computer interaction.

With the widespread application of LLMs in dialogue scenarios, traditional dialogue analysis struggles to capture deep cognitive processes (e.g., task planning, metacognitive monitoring). Google DeepMind's 2026 paper Measuring Progress Toward AGI: A Cognitive Framework proposes that AGI evaluation should be based on a cognitive science framework, decomposing intelligence into measurable cognitive dimensions and providing a new methodology for dialogue analysis.

The plugin's core is a coordinator + four specialized sub-agent architecture, each responsible for one cognitive dimension:

Agent	Annotation Dimension	Extracted Content
Executive Functions	Executive Function Behaviors	Planning, inhibition, switching behaviors
Metacognition	Metacognitive Processes	Knowledge boundary awareness, confidence calibration, etc.
Memory & Reasoning	Memory & Reasoning Patterns	Domain knowledge injection, various reasoning types
User Mental Model	System Mental Model	Model updates, collaboration strategies

The system architecture includes an entry layer, coordination layer, execution layer, and merging layer; the execution flow is user call → parse records → invoke sub-agents → merge results. Annotation follows four core principles: lenient extraction, trivial substitution, no pattern filling, and annotation only for human turns.

The plugin's theory is derived from classic cognitive science research:

• Executive Functions: Miyake et al.'s 2000 update/inhibition/switching model;
• Metacognition: Nelson & Narens' 1990 monitoring and control framework;
• Memory and Reasoning: Peirce's reasoning types + Gentner's 1983 structure mapping theory;
• User Mental Model: Norman's 1983 mental model theory for interactive systems.

The plugin can be applied in:

1. Academic research: Providing structured dialogue cognitive data for AGI evaluation;
2. Product optimization: Helping AI teams understand user interactions and optimize dialogue design;
3. Educational technology: Analyzing interactions between learners and AI tutoring systems, and evaluating metacognitive strategies;
4. Clinical assistance: A standardized dialogue cognitive analysis tool for cognitive assessment and rehabilitation.

The plugin is a native Claude Code form, with an additional Python version (using the Claude Agent SDK to implement parallel execution and write to results.json). Usage recommendations:

• Use the Claude Code plugin for interactive annotation;
• Use the Python pipeline for batch processing/automation.

The cognitive-annotation-plugin combines cognitive science theory with LLM capabilities to build a scalable and reproducible cognitive analysis tool, which is an important cornerstone for the standardization of AGI evaluation. As human-computer interaction deepens, understanding dialogue cognitive processes will become the core of AI research. This plugin provides researchers with an out-of-the-box scientific tool to help explore the nature of intelligence.