AgentMemoryManager: A Four-Layer Cognitive Memory Architecture for LLM Agents

AgentMemoryManager is an LLM agent memory management component inspired by human memory. It effectively addresses the context degradation issue in long conversations through a four-layer architecture (working memory, episodic memory, semantic memory, and procedural memory). It supports multiple storage backends (e.g., Chroma/Qdrant, SQLite) and LLM providers (e.g., Ollama, OpenAI), enhancing agent performance and user experience.

With the widespread use of LLMs in agent applications, the context degradation issue has become increasingly prominent: as the number of conversation turns increases, the ability to remember early information drops sharply (the accuracy of buried-in-the-middle information decreases by over 30%), token costs grow linearly, and cross-session memory is completely lost. Traditional solutions (truncating history, periodic summarization) either lose important information or fail to capture details, restricting the performance of agents in complex tasks.

Four-Layer Memory Architecture

• Working Memory: Manages the immediate context of the current session, using compression and sliding window techniques to retain key information
• Episodic Memory: Stores atomized facts extracted from conversations, enabling cross-turn memory
• Semantic Memory: Builds an entity-relationship knowledge graph to support reasoning and association
• Procedural Memory: Saves reusable task templates and tool usage patterns

Technical Implementation

• Multiple Memory Strategies: Sliding window, summary generation, atomic fact extraction, reflection mechanism, Zettelkasten
• Multi-Backend Storage: InMemory, SQLite, Chroma/Qdrant, PostgreSQL+pgvector
• Multi-LLM Compatibility: Anthropic Claude, OpenAI GPT, Ollama, LiteLLM
• Framework Integration: LangChain, LlamaIndex, Custom Agent (Python SDK)

Performance Benchmarks (ACL 2024 LOCOMO Test)

Solution	Accuracy	P95 Latency	Tokens per Session
Full Context (Baseline)	72.9%	9.87s	~26,000
AgentMemoryManager	≥65%	<2s	<4,000
Key Insight: Accuracy remains at an acceptable level, latency is reduced by 5x, and cost is optimized by approximately 85%.

Academic Support

Based on cutting-edge research from 2023-2025: Mem0 (atomic fact extraction), Generative Agents (reflection mechanism), A-MEM (Zettelkasten linking), StreamingLLM (attention management), LLMLingua (token compression).

Practical Application Value

• Enhance user experience: Remember user preferences and historical interactions, provide personalized continuous services
• Reduce operational costs: Token consumption reduced by 85%, lowering API call costs
• Enhance system capabilities: Support long conversations, multi-session interactions, and complex tasks
• Protect data privacy: Support fully local deployment

Production-Ready Features

• Structured logging: Facilitates debugging and monitoring
• Prometheus metrics: Integrate with monitoring systems
• GDPR-compliant deletion: Meet privacy regulation requirements

• v1.5 (In Progress): Neo4j backend support, automatic entity extraction, knowledge graph querying
• v2.0 (Planned): PGVector integration, streaming compression, multi-modal memory support

AgentMemoryManager provides an elegant and practical solution to the memory management problem of LLM agents through its human-like four-layer memory architecture. It solves the context degradation problem, and its modular design supports multi-scenario applicability, making it a tool worth the attention and trial of agent developers.