Aurora: A Privacy-Focused Localized Smart Voice Assistant

Core Points: Aurora is an open-source smart voice assistant focusing on local privacy protection and productivity improvement. All processing is done locally. It integrates real-time speech-to-text, large language models, and various open-source tools. This thread will introduce its background, features, architecture, installation, vision, etc., in separate floors to help everyone fully understand this privacy-first AI assistant.

• Original Author/Maintainer: joaojhgs
• Source Platform: GitHub
• Original Link: https://github.com/joaojhgs/aurora
• Release Date: February 10, 2025
• Last Updated: June 15, 2026

Aurora's core concept is a 'privacy-first Swiss Army knife assistant'. All processing is done locally to ensure data never leaves the device. It is developed in Python (supports versions 3.10-3.11), follows the MIT open-source license, currently has 13 stars, and is in the early stage but has significant potential in architecture design and function planning.

1. Wake Word Detection

Supports custom wake words (e.g., 'Jarvis'). OpenWakeWord enables offline low-latency wake-up without network connection for activation.

2. Real-Time Speech-to-Text

Uses the Whisper model for real-time speech-to-text, including an 'ambient transcription' feature. Background continuous recording and transcription support daily activity summaries.

3. Large Language Model Integration

Supports multiple providers (OpenAI, HuggingFace Pipeline/Endpoint, Llama.cpp); locally supports quantized models like Llama3 and Mistral7B; can remotely connect to HuggingFace inference endpoints; parameter management via JSON configuration.

4. Semantic Search and OpenRecall Integration

Regularly takes screenshots to index user activities, enabling semantic historical record retrieval (e.g., querying 'the interface researched at 2 PM').

5. Text-to-Speech

Piper enables offline TTS to generate natural voice responses.

6. MCP Support

Connects to external MCP servers to expand capabilities, supports local (stdio) and remote (HTTP) servers, dynamic tool loading and authentication.

Modular plugin architecture, prioritizing privacy, scalability, and local processing. Core components:

1. Configuration Management: config_manager.py centrally handles settings; config.json and .env separate sensitive credentials.
2. Audio Processing Pipeline: OpenWakeWord wake detection, Whisper real-time STT, threaded architecture ensures UI responsiveness.
3. LangGraph Orchestration: Intelligently routes LLM inference and tool execution; RAG-based tool selection; maintains conversation context.
4. Plugin System: Independent plugins, conditional loading, extensible (add new tools without modifying the core).
5. Memory and Storage: Vector storage supports semantic search; SQLite stores conversation history and system state.

Installation Methods

1. Docker Hub: Pre-built images for quick deployment; execute relevant docker pull and docker-compose commands.
2. UV Installation: Recommended for developers; fast dependency resolution; requires git clone of the project followed by uv sync and run commands.
3. Source Code Installation: Guided setup via setup.sh (Linux/macOS) or setup.bat (Windows); automatically checks environment and installs dependencies.

Model Management

• Chat Models: Stored in chat_models/ (GGUF format, 2-4GB); configure path in config.json; can be downloaded from HuggingFace GGUF library.
• Voice Models: Stored in voice_models/ (Piper and wake word models); after configuring the path, more voices can be downloaded from Piper Voices.

Client-Server Architecture

• The server receives and processes audio; clients can have local tools and be called by the server; supports low-cost devices like ESP32; WebRTC enables peer-to-peer connections.

Smart Home Integration

• Supports integration with smart home devices; controls smart appliances via tool calls.

Core Vision: Allow users to interact via low-cost interfaces in a private network; the assistant can control real devices or multiple desktops.

Aurora represents an important direction for privacy-first voice assistants. Its local-first design, modular plugin architecture, and flexible LLM support make it a promising open-source project. It is suitable for users and developers who care about privacy and want to run AI assistants locally, providing a feature-rich and complete solution. With the implementation of client-server architecture and smart home integration, it is expected to become an important tool in the fields of home automation and personal productivity.