# Mithrandir: A Complete Guide to Building a Local Privacy-First AI Assistant Based on Gemma 4

> Mithrandir is an open-source local AI assistant project that demonstrates how to build a complete agentic system on consumer-grade hardware, using Gemma 4 for local inference and Claude as a backup inference engine.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-04-25T06:09:17.000Z
- 最近活动: 2026-04-25T06:23:31.975Z
- 热度: 169.8
- 关键词: Mithrandir, Gemma 4, 本地AI, 隐私优先, Agentic系统, Ollama, Claude, RAG, 语音克隆, 本地部署, ReAct, EDGAR, 量化分析
- 页面链接: https://www.zingnex.cn/en/forum/thread/mithrandir-gemma-4ai
- Canonical: https://www.zingnex.cn/forum/thread/mithrandir-gemma-4ai
- Markdown 来源: floors_fallback

---

## Mithrandir Project Introduction: Guide to Building a Privacy-First Local AI Assistant

Mithrandir is an open-source local AI assistant project with a core focus on privacy-first and local hosting. It uses Gemma 4 for local inference (with Claude as a backup engine) and aims to replicate the capabilities of cloud-based AI assistants on consumer-grade hardware. This project is not just usable software but also a detailed build log with comprehensive documentation, providing learners with a reference for the actual building process.

## Project Positioning and Core Philosophy

The project name is derived from Gandalf in *The Lord of the Rings*, symbolizing wise guidance. Its core positioning is a privacy-first, locally hosted AI assistant, emphasizing building a 'system that runs LLMs' rather than the LLM itself to lower the entry barrier: no need to understand the mathematical details of Transformers—just integrate existing models into the application stack.

## Overview of Hybrid Technical Architecture

Adopts a hybrid local+cloud architecture:
- Local inference: Gemma 4 26B (MoE) runs via Ollama, achieving 144 tokens/sec on RTX4090 (4x faster than cloud APIs)
- Cloud backup: Complex tasks are routed to the Claude API
- Agent framework: ReAct loop + Pydantic tool calling
- Memory system: ChromaDB (vector database) + SQLite (long-term memory + code RAG)
- Market module: HMM market state recognition + EDGAR quantitative screening (covers 9.8K SEC filings)
- Voice interaction: faster-whisper recognition + F5-TTS cloning + Kokoro TTS
- UI: React/FastAPI/WebSocket frontend + Telegram bot

## Hardware Configuration and Model Selection Guide

Hardware requirements are user-friendly:
| Component | Minimum Configuration | Recommended Configuration |
|---|---|---|
| GPU | NVIDIA 8GB VRAM | NVIDIA 20GB+ VRAM |
| Memory | 16GB | 32GB+ |
| Storage | 50GB | 100GB+ |
| OS | Win10/11 or Linux | Win11 or Ubuntu|
Model selection: Gemma4 26B runs on 24GB VRAM; the e4b variant (4B active parameters) runs on 8GB VRAM.

## Core Features and Application Scenarios

Core capabilities include:
- SEC filing analysis: Reads public company filings to assist fundamental research
- Market tracking: Real-time data + HMM market state recognition
- Persistent memory: Maintains dialogue context coherence
- Voice cloning: Uses F5-TTS technology for voice cloning
- Codebase RAG: Understands personal codebases to provide programming assistance

## Learning Value and Comprehensive Documentation

The project documentation has great learning value:
- JOURNEY.md records the complete build process (steps, bugs, solutions)
- Key FAQ questions:
  1. Accessible to ordinary users (requires NVIDIA GPU with 8GB+ VRAM)
  2. Cost is only electricity (zero API fees)
  3. Performance on daily tasks is comparable to cloud-based assistants; Claude is better for complex reasoning
  4. Data does not leave the device by default

## Deployment Steps and Access Methods

The deployment guide is complete, taking an estimated 1-2 hours (mainly for model downloads). Access methods: Browser (custom frontend) or iPhone (Telegram bot).

## The Significance of Mithrandir for AI Democratization

It represents the direction of AI democratization: enabling individuals to run powerful AI systems on local hardware without relying on cloud service providers. The improved capabilities of open-source models (Gemma/Llama/Qwen) plus advances in consumer-grade GPUs make local AI more practical. As a complete reference implementation, this project lowers the barrier for developers to enter the agentic AI field.