Docker AI Stack: The Ultimate Solution for One-Click Deployment of a Complete Local AI Tech Stack

A complete local AI tech stack based on Docker Compose, integrating Ollama, LiteLLM, Whisper, Kokoro, Embeddings, and MCP Gateway. It supports GPU acceleration and provides end-to-end AI capabilities from voice input to voice output.

The design philosophy of docker-ai-stack is "zero configuration" and "privacy first". It integrates the most popular open-source AI services currently available, enabling rapid deployment and isolated operation of services via Docker containerization technology. All core services run locally, and data is not sent to third parties, making it particularly suitable for scenarios with strict data privacy requirements.

docker-ai-stack includes six core services, covering the complete AI pipeline from input processing to output generation:

• Role: Runs local LLM models (e.g., llama3, qwen, mistral, etc.)
• Default Port: 11434
• Features: Supports multiple open-source models, GPU-accelerated inference

• Role: Unified API gateway that routes requests to Ollama or over 100 external providers
• Default Port: 4000
• Features: OpenAI-compatible API format, supports model load balancing and failover

• Role: Converts text into vectors, supports semantic search and RAG applications
• Default Port: 8000
• Features: Runs locally, no external API required

• Role: Transcribes voice audio into text
• Default Port: 9000
• Features: Supports multiple languages, local processing protects privacy

• Role: Converts text into natural speech
• Default Port: 8880
• Features: High-quality speech synthesis, supports multiple voices