Monk: Cost Leak Detection Tool for AI Agent Workflows

Monk is an open-source tool designed to identify hidden cost waste and blind spots in AI agent workflows. By analyzing LLM call trace data, it detects issues like repeated calls, overuse of expensive models, and more, helping developers significantly reduce operational costs.

With LLM deployment in production, teams often face unexpected API cost increases. The problem isn't single expensive calls but hidden inefficiencies: repeated tool calls, improper model selection, invalid retries—these 'chronic leaks' can cost tens to hundreds of dollars daily.

Developed by Blueconomy AI, Monk is a blind spot detector for agentic workflows. Unlike traditional observability tools (which focus on dashboards), Monk uses trace files to identify 5 cost waste patterns and provides actionable fix suggestions, directly addressing how money is wasted and how to stop it.

Monk has 5 built-in detectors:

1. Retry Loop: 3+ consecutive calls to the same tool (e.g., repeated web searches without strategy changes).
2. Empty Return Trap: Null/empty tool results fed to LLM (wastes tokens and causes hallucinations).
3. Model Overkill: Using expensive models (like GPT-4o) for simple tasks (e.g., classification), with potential 16x cost reduction by switching to lighter models.
4. Context Bloat: System prompts taking over 55% of token budget or untruncated history leading to rising input tokens.
5. Agent Loop: Smart agents stuck in the same step sequence without progress.

Installation: pip install monk-ai Commands:

• Analyze single trace file: monk run agent_traces.jsonl
• Analyze directory: monk run ./traces/
• Run specific detectors: monk run traces/ --detectors retry_loop,model_overkill
• Export JSON report: monk run traces/ --json findings.json
• Show high-severity issues: monk run traces/ --min-severity high Supported Formats: OpenAI Chat Completions, Anthropic Messages, LangSmith exports, and custom JSONL (with session_id, model, input_tokens, output_tokens, tool_name, tool_result fields).

A real analysis example: 2,847 calls analyzed, found 3 blind spots causing $62.4/day ($1,872/month) waste. Issues included:

1. Retry loop (web_search called 4x in row: ~$38.2/day)
2. Empty returns from get_user_profile (80% empty: ~$19.1/day)
3. Model overkill (GPT-4o for simple tasks: ~$5.1/day) Fixes: Add max-retries guard, check for empty returns, switch to GPT-4o-mini—leading to monthly savings of nearly $2,000.

Monk is developed by Blueconomy AI (Techstars 2025 member) and licensed under MIT (fully open-source, community contributions welcome). Future plans:

• Real-time monitoring via OpenTelemetry
• Prompt compression suggestions
• Cross-workflow performance benchmarks
• Slack/PagerDuty alert integration
• Web management dashboard.

Monk is a lightweight, efficient cost audit tool for production LLM apps—no architecture changes needed (just export trace data). Recommended use cases:

• Diagnose when monthly API bills exceed expectations
• Baseline efficiency testing before new feature launches
• Add cost regression checks in CI pipelines For scaling AI apps, cost control is as important as model capability—Monk helps balance both.