Hermes Architect Audit: An Open-Source Solution for AI Cost Optimization via Intelligent Routing

Hermes Architect Audit (HAA) is an automated AI cost intelligence auditing tool that addresses the "model paradox" in enterprise AI engineering—significantly cutting operational costs while ensuring top-tier reasoning capabilities—through an intelligent task routing framework. Its core innovation lies in dynamically assigning tasks to the optimal model, which can reduce daily task costs by 99% while maintaining high-performance output, providing an open-source cost optimization solution for multi-agent workflows.

Current enterprise AI tech stacks often "use the best model for all tasks", even deploying flagship models like Claude Opus and GPT-4o for simple tasks (e.g., conversational search, approval checks), leading to severe budget waste. For example: GPT-5.5 ($5/$30 per million tokens) for conversational search, Opus4.6 ($5/$25) for approval checks, etc. The root cause is a routing problem—all tasks are routed to flagship models, resulting in uncontrolled costs.

The core innovation is the A vs B task routing framework, where each task node evaluates two options: Option A (efficiency leader, low cost with acceptable quality) and Option B (performance leader, highest quality with acceptable cost). The technical architecture is layered: config.yaml defines task slots and model assignments → LLM Analysis performs cost/performance analysis via the OpenRouter API → Python FPDF renders the report → PDF Report outputs in A4 landscape format. The architectural advantage is separation of concerns, avoiding LLMs from simultaneously handling reasoning and presentation code.

HAA automatically detects over-configuration and recommends models. Zero-cost task slots: 6 daily tasks (conversational search, approval checks, etc.) use the free Llama-3.2-3B model, reducing costs to zero while achieving latency <100ms or equivalent quality. Hierarchical routing strategy: Heavy tasks (long context, complex reasoning) → Gemini3.1 Flash Lite ($0.25/$1.50, 1M context); Medium tasks → DeepSeek-V3.1 ($0.15/$0.75); Light tasks → free 3B models.

Cost-effectiveness: 6 task slots are zero-cost, and the remaining 5 use cost-effective models, saving $100-150 per month and reducing costs by 99%. Performance metrics: Input cost from a maximum of $5 → $0 for 6 slots; Output cost from a maximum of $30 → $0 for free tiers, $1.5 for heavy tasks; First-token latency from over 800ms → <100ms; Context window from fragmented 128K → seamless 1M+.

Engineering lessons: Avoid having LLMs handle both reasoning and generating presentation code (e.g., HTML/PDF), as this leads to hallucinated layouts, resource waste, and maintenance difficulties. HAA's solution is strict separation of concerns. Applicable scenarios: Multi-agent workflows, cost-sensitive applications, high-frequency call scenarios, teams with hybrid model strategies. The savings effect amplifies as query volume grows.

HAA demonstrates the trend in AI engineering: shifting from one-size-fits-all model selection to refined task routing. Core values: Practicality (addresses the pain point of uncontrolled costs), implementability (complete toolchain), scalability (supports custom tasks and models). It provides a reference for AI infrastructure teams to help balance quality and cost.