Zeph: A Production-Grade Rust AI Agent Framework (Overview)

Zeph is a high-performance AI Agent framework written in Rust, designed for production environments. It addresses key pain points of existing Agent frameworks with advanced features like hybrid reasoning, self-learning skills, temporal graph memory, cascaded quality routing, and OWASP AI security reinforcement. Its design philosophy emphasizes maximizing the value of every context token, making it suitable for long-running, resource-efficient agent services.

Key keywords: AI Agent, Rust, LLM, MCP, context compression, hybrid reasoning, graph memory, OWASP, security, ReAct.

Background & Motivation: Addressing Production Agent Pain Points

As LLMs evolve, AI Agents are moving from experimental tools to production, but existing frameworks face core challenges: inefficient context window management, complex multi-model switching, insufficient security considerations, and bloated runtime dependencies. Zeph was born to solve these pain points as a Rust-based single-binary AI Agent.

Rust was chosen for its zero-cost abstractions and memory safety, ensuring high performance and reliability. Zeph’s single binary is ~15MB, starts in ~50ms, and uses ~20MB idle memory—critical for long-running agent services.

Hybrid Inference Architecture: Flexible & Cost-Efficient Multi-Model Orchestration

Zeph supports multiple LLM providers (Ollama, Claude, OpenAI, Google Gemini, OpenAI-compatible endpoints, and local GGUF models via Candle). Its multi-model orchestration includes:

• Cascaded routing & cost optimization: Explicit cost-tiered routing (cheapest first) to avoid overusing expensive models for simple queries.
• Complexity triage: LlmRoutingStrategy::Triage classifies queries into 4 levels (simple/medium/complex/expert) and dispatches to corresponding provider pools.
• PILOT LinUCB: Context-aware LinUCB algorithm for dynamic provider selection (considering query complexity, historical latency, time signals).
• EMA delay routing: Exponential moving average-based latency prediction with adaptive Thompson sampling for balanced exploration/exploitation.

Skills-first Architecture: Dynamic, Self-Learning Agent Capabilities

Zeph uses a skills-first architecture where skills are defined via YAML+Markdown files, supporting BM25 + cosine similarity hybrid retrieval for dynamic loading.

Key self-learning features:

• Bayesian reordering and 4-level trust model for skill improvement from usage.
• Agent-as-a-Judge feedback detection (supports 7 languages: English, Russian, Spanish, German, French, Portuguese, Chinese) with adaptive regex + LLM hybrid analysis.
• On-demand skill loading: LLM can load full skill content via load_skill tool when needed, balancing context brevity and scalability.

Context Engineering & Memory Systems: Maximizing Token Value

Zeph’s context engineering focuses on every token’s value with a 3-layer compression pipeline:

• Delay application: Trigger summary at 70% context usage, pruning at 80%, LLM compression on overflow.
• HiAgent subgoal-aware compression: Protects active subgoal messages, summarizes completed ones.
• ACON failure-driven compression: Learns from context loss failures to generate compression guides.
• Memex tool output archiving: Stores large outputs in SQLite (not disk) for on-demand injection via read_overflow.

Memory features:

• SQLite/PostgreSQL+Qdrant backend with MMR reordering, time decay, importance scoring, and query-aware routing.
• Graph memory: Entity relation tracking (8 types), FTS5 search, BFS multi-hop reasoning, dual-temporal versioning, SYNAPSE diffusion activation, and A-MEM dynamic note links.
• RL admission control: Logic regression model for memory writes (falls back to heuristics if insufficient samples).

Security Reinforcement: OWASP AI Agent 2026 Compliance

Zeph implements OWASP AI Agent Security 2026 measures:

• Deep defense: Shell sandbox, SSRF protection, skill trust isolation, key zeroing, audit logs, and unsafe_code = "deny" policy.
• Untrusted content isolation: ContentSanitizer processes tool results, web scrapes, MCP responses, etc., with truncation, control character stripping, 17 injection pattern detection, and XML delimiter wrapping.
• PII filter: Desensitizes emails, phones, SSNs, credit cards, and custom patterns using zero-allocation Cow paths.
• Memory write validator: Enforces size limits, substring bans, entity/edge limits, and PII scans.
• Tool rate limiter: Sliding window-based category limits with circuit breaking and atomic slot reservation to prevent parallel bypass.

Integration & Ecosystem: IDE, MCP, LSP & Task Orchestration

Zeph supports:

• ACP protocol: Stdio, HTTP+SSE, WebSocket for multi-session isolation and SQLite persistence (works in Zed, Helix, VS Code).
• Multi-channel I/O: CLI, Telegram, TUI dashboard (all streaming), plus voice/visual input.
• MCP client: Cleans tool definitions (17 injection checks, Unicode Cf stripping, 1024-byte description limit) to prevent prompt injection.
• LSP context injection: Auto-injects compiler diagnostics after file writes, prefetch hover info after reads, and lists call sites before renaming (supports 30+ LSP servers like rust-analyzer).
• Sub-agents: Isolated with scoped tools/skills and zero-trust key delegation (Markdown definitions with 4-level priority).
• Task orchestration: DAG-based task graphs with dependency tracking, parallel execution, configurable failure strategies, and SQLite persistence (LLM-driven goal decomposition via Planner trait).

Summary & Outlook: Production-Ready AI Agent Framework

Zeph represents a step toward production-grade AI Agent maturity. It solves key pain points via Rust’s performance, fine-grained context engineering, multi-layer security, and flexible hybrid reasoning.

For developers building reliable, efficient, secure agents, Zeph is a valuable reference. Its single-binary model fits containerized/edge deployments, and protocol support (MCP, A2A, ACP) ensures ecosystem interoperability.

Detailed docs are available at bug-ops.github.io/zeph for deeper understanding of its design philosophy.