CodeClue: A Persistent Code Understanding System for LLMs with Confidence-Guided Source Code Drilling Mechanism

CodeClue is an innovative code understanding system designed for LLMs. It generates persistent "clue files" (graph-structured code understanding products) to optimize LLM-codebase interactions. Key achievements include an 81% token reduction compared to raw source code approaches, zero hallucination rate, and a confidence scoring mechanism that intelligently decides when to drill into source code. It uses the Model Context Protocol (MCP) to expose tools for AI assistants to access deeper information as needed.

In LLM-assisted programming, a long-standing issue is repeated reasoning over codebases—each interaction requires re-reading and understanding large amounts of source code, even for previously analyzed parts. This wastes tokens, computing resources, and increases response latency. CodeClue addresses this by introducing persistent clue files (typically 5x smaller than raw code) and a confidence-driven approach to only access source code when necessary.

CodeClue uses a two-layer code reference system:

1. Structural Tier (Tier1): Always present, derived from AST parsing and regex. Includes basic symbol info (purpose, name, type), call relationships, and complexity metrics. Ensures zero hallucination and answers architecture overview questions.
2. Semantic Tier (Tier2): Generated by LLMs, includes deeper insights like pre/post conditions and failure modes. Triggered based on confidence scores.

Confidence scoring considers coverage gaps, dependency closure, and code density risk. The system exposes 5 MCP tools: code_slice (get source lines), resolve_dependency (expand dependency subgraph), check_freshness (compare clue vs source hash), expand_projection (extend node view), fetch_contract (get semantic contracts).

CodeClue was tested on 7 public codebases (Flask, FastAPI, NestJS, httpx, Express, TypeORM, Gin) across 4 languages (Python, TypeScript, JavaScript, Go) with 23 tasks. Results:

• Token reduction: 81% vs raw source-first approach.
• Hallucination rate: Zero across all tasks, codebases, and model families (Claude, GPT, Gemini).
• Confidence accuracy: IFT alignment of 0.65 (low confidence correctly predicts need for drilling).
• Cross-model validity: Average difference of only 0.12 between models.

Highlights:

• Persistent understanding: Cachable, versionable clue files enable cross-session/user sharing.
• Confidence-guided interaction: Explicitly tells LLMs its confidence level and when to verify.
• MCP protocol application: Standardized tool interaction for AI assistants.

Limitations:

• Upfront cost for clue generation (long initial processing for large repos).
• Clue files may become stale when source code changes (needs periodic refresh).
• Confidence scoring isn't perfect (edge cases of over/under estimation).

Future Directions:

• Incremental update mechanism (re-analyze only changed parts).
• Finer-grained confidence dimensions.
• Support for more programming languages.
• Deep integration with IDEs.

Conclusion: CodeClue represents a significant advancement in code understanding for LLMs. By combining persistent clue files and confidence-guided drilling, it balances efficiency (token reduction) and accuracy (zero hallucination), providing a scalable, verifiable, and efficient paradigm for AI-assisted programming with large codebases.