Aigate: Multi-vendor AI Gateway's Self-Healing Architecture and Free-Priority Routing Strategy

Aigate is a Docker Compose-based multi-vendor AI gateway that integrates dozens of AI vendors into an OpenAI-compatible endpoint. It features intelligent failover, a cost-optimization strategy with free-priority routing, and provides dual-instance Claude Code agent capabilities along with complete auxiliary services, solving the risks of single-vendor dependency and development complexity.

The large language model market is highly fragmented, with OpenAI, Anthropic, Google, open-source model hosting platforms, and inference service providers each having their own advantages. A single-vendor strategy carries risks such as service outages, price changes, and rate limits. Production-grade applications require multi-vendor redundancy but face increased development complexity.

Aigate uses Docker Compose to deploy a complete tech stack. Core components include: Nginx (unified entry gateway, port 4000), LiteLLM Proxy (core proxy layer, providing OpenAI-compatible API, load balancing, etc.), PostgreSQL (key management, budget tracking, usage statistics), Redis (response caching and rate limiting), dual-instance Claude Code (connecting to Anthropic's official API and z.ai GLM model), HybridS3 (S3-compatible object storage), and Stealthy Auto Browse (a cluster of 5 browser replicas). All services are exposed via Nginx port 4000 and routed to different backends based on path prefixes.

Aigate integrates mainstream AI vendors: Groq (ultra-fast inference, 1 million free tokens daily), Cerebras (wafer-scale chips, 50 free requests daily), OpenRouter (aggregation platform), HuggingFace (open-source model hub), Anthropic/OpenAI (official APIs), and z.ai (Zhipu AI). It also defines unified model aliases to simplify usage, e.g., groq-llama-3.1-8b maps to llama-3.1-8b-instant.

Aigate predefines model groups (Fast/Smart/Vision/Image-gen/Transcription, etc.) and routes based on priority: Groq free quota first → Cerebras free tier → OpenRouter free models → paid APIs. It automatically fails over when the preferred model is unavailable, with the process transparent to clients. This strategy balances service quality and operational costs.

The dual-instance Claude Code (connecting to Anthropic's official API and z.ai GLM) has full CLI capabilities: file operations, shell execution, tool usage, and multi-round collaboration, suitable for scenarios like code review and automated refactoring. Auxiliary services include HybridS3 (storing generated content and multimodal inputs) and Stealthy Auto Browse (web scraping and automated testing), which support MCP interfaces for Claude Code to call.

Production features: Key management (centralized storage in PostgreSQL), rate limiting (distributed limiting via Redis), budget control; observability (usage statistics, latency monitoring, error tracking); caching strategy (Redis reuses responses for identical inputs). Deployment steps: git clone https://github.com/psyb0t/aigate → cd aigate → docker compose up -d, configure API keys via environment variables (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.).

Ideal scenarios: Multi-tenant SaaS, cost-sensitive applications, high availability requirements, model experiments. Notes: Latency from free vendors, differences in model output styles, compliance considerations, vendor lock-in risks. Conclusion: Aigate reduces AI integration complexity; its free-priority strategy is suitable for startup teams, making it a reference architecture and best practice for production-grade AI systems.