Enkidu: An Open-Source Local AI Assistant Project Based on Gemma 4 and Claude API

Enkidu is an open-source local AI assistant project that combines Google's local Gemma4 model with Anthropic's Claude API as a fallback, supporting RTX4090 CUDA-accelerated inference. It provides a complete practical case for learning Agentic systems, GPU computing, and full-stack LLM deployment, with both data privacy protection and complex task handling capabilities.

With the development of large language model technology, locally deployed AI assistants have gained attention due to privacy protection and reduced API costs. The name Enkidu comes from a character in the Epic of Gilgamesh, symbolizing the partnership between AI and human wisdom, and aims to help developers understand the core concepts of Agentic systems, GPU computing, and full-stack LLM deployment.

Enkidu adopts an intelligent model scheduling strategy:

1. Primary Model: Gemma4 – Google's open-source lightweight model that can run locally on RTX4090, supports CUDA-accelerated inference, and works without a network;
2. Fallback Model: Claude API – Automatically switches when the local model cannot handle complex tasks, using Claude's powerful reasoning capabilities to ensure continuous experience. This architecture balances privacy, response speed, and complex scenario handling capabilities.

Enkidu fully leverages the advantages of the RTX4090 graphics card: 24GB GDDR6X memory, 16384 CUDA cores, and 4th-generation Tensor cores to support efficient inference. Optimization strategies include: model quantization to reduce memory usage, dynamic batching to improve GPU utilization, KV cache optimization to reduce redundant computations, memory management to avoid OOM errors, and streaming responses for word-by-word output.

Agentic Capabilities: Supports file system operations, code execution, network requests, and system commands; has task planning (subtask decomposition, state maintenance, dynamic adjustment) and efficient context management (sliding window, importance scoring, long document summarization). Full-Stack Deployment: The backend includes a model service layer (vLLM/TGI), API gateway, business logic layer, and data storage layer; the frontend provides a clean chat interface, Markdown rendering, and file upload/download; supports local development, Docker containerization, and cloud expansion deployment methods.

Learning Value:

• For developers: End-to-end implementation reference, performance optimization techniques, architecture design patterns, and troubleshooting experience;
• For AI learners: Local LLM deployment, GPU programming basics, Agentic system principles, and full-stack development workflow;
• For privacy users: Local processing of sensitive data, open-source audibility, and no data collection. Community and Future: Contributions to model support, tool expansion, UI improvements, and documentation refinement are welcome; future plans include multimodal input, RAG knowledge base, voice interaction, and mobile optimization.

Enkidu represents an important direction for AI applications: utilizing local computing resources under privacy protection. With the improvement of open-source model capabilities and hardware advancements, local AI assistants will become more practical. For developers, Enkidu is both a tool and a learning platform, helping to master the complete technology stack from CUDA optimization to Agentic system design. Whether you are a privacy user or a learner, Enkidu is worth paying attention to and trying.