Empirica: A Measurement System for AI Agents to Gain Self-Awareness

Empirica is a cognitive measurement system designed to solve core issues of AI coding assistants: lack of self-awareness, inter-session forgetting, acting before understanding, and inability to distinguish between knowledge and speculation. Through 13-dimensional vector confidence assessment, the Sentinel access control mechanism, and a cross-session memory system, it helps AI agents measure their own knowledge state, verify their level of understanding before acting, and continuously accumulate learning to improve reliability and predictability.

Current AI coding assistants have fundamental flaws—lack of self-awareness—leading to the following issues:

• Inter-session forgetting: Starting from scratch in each new session, repeating questions and mistakes
• Acting before understanding: Modifying code without understanding the codebase architecture
• Inability to distinguish between knowledge and speculation: Failing to clearly inform users of speculative vs. confident situations
• Lack of audit trails: Reasoning processes disappear as the context window refreshes Empirica, as a "cognitive measurement system", was created precisely to address these problems.

The core of Empirica is to "give AI a mirror". Its main mechanisms include:

1. Cognitive Vector: A 13-dimensional confidence assessment system (foundation layer, understanding layer, execution layer, metacognition layer) that displays cognitive status in real time
2. Cognitive Transaction Cycle: Noetic-Praxic cycle (PREFLIGHT assessment → CHECK access control verification → POSTFLIGHT learning outcome persistence)
3. Four-Layer Memory System: Working memory, short-term memory, long-term memory, and external memory to solve context limitations
4. Sentinel Access Control: Verify cognitive vector thresholds before acting to prevent blind editing
5. Calibration System: Compare self-assessments with objective results (tests, Git metrics, etc.) using Brier scores to achieve a positive cycle

Empirica is deeply integrated with Claude Code, providing features like automatic hooks, access control, and status bar displays; it also supports Cursor, Cline (via MCP server), Gemini CLI, and Copilot (experimental). Installation methods:

• pip: pip install empirica && empirica setup-claude-code
• Homebrew (macOS): brew tap nubaeon/tap && brew install empirica After configuration, the measurement system runs automatically in the background.

Empirica adopts a fully local-first design:

• Local SQLite database (.empirica/)
• Git cognitive checkpoints (.git/refs/notes/empirica/*)
• Local Qdrant vector database No cloud dependencies, no telemetry data, and users have full control over their cognitive data.

Empirica represents the evolution direction of AI-assisted development tools—shifting from pursuing code generation capabilities to reliability and predictability. Through cognitive science measurement methods, it enables AI coding assistants to have self-awareness, helping developers understand the AI's level of confidence and make informed decisions. For developers dealing with complex codebases and valuing quality and maintenance, "measurable reliability" is a key step toward the maturity of AI-assisted development.