AgentTrace: A Solution for Observability and Cost Tracking of AI Agent Workflows

AgentTrace is an open-source observability SDK for AI agents, designed specifically to address debugging difficulties, cost opacity, and performance bottlenecks in complex agent workflows. It provides full execution tracing, cost attribution, real-time log streaming, and waterfall playback features, is compatible with the OpenTelemetry standard, and supports self-hosted deployment. This article will cover its background, architecture, features, usage methods, and more.

Modern AI agent workflows involve complex processes such as multiple LLM calls, tool executions, and decision branches, leading to three core issues: opaque execution processes (difficulty understanding decision logic), untraceable costs (blurred expenses across multiple links), and hard-to-reproduce problems (lack of diagnostic context). Existing tools like LangSmith and Langfuse are insufficient in key functions such as cost tracking and real-time log streaming.

AgentTrace adopts a modular architecture with three core components:

1. SDK Layer (Python library): Provides decorators (e.g., trace_openai) and hybrid clients, supporting switching between simulated and real calls;
2. Server (FastAPI): Offers cost analysis, budget tracking, WebSocket real-time log streaming, and supports SQLite/PostgreSQL databases;
3. Visual Dashboard (Next.js): Displays data visualization features such as run lists, cost details, and waterfall playback.

AgentTrace's key features include:

• Fine-grained cost tracking: Span-level analysis, statistics by model/tool/workflow dimensions, and built-in budget alerts;
• Real-time log streaming: Monitor agent operation status in real time via WebSocket;
• Waterfall playback and prompt replay: Fully display the time consumption of execution steps and support reproducing decision processes;
• Multi-agent correlation tracking: Realize cross-agent call chain tracking via correlation_id;
• Batch ingestion API: Efficiently import large amounts of tracing data to adapt to high-throughput scenarios.

Quick Deployment Steps:

1. Clone the repository and install the SDK: cd sdk && pip install -e .
2. Start the server: cd server && pip install -r requirements.txt && uvicorn app.main:app --reload
3. Start the dashboard: cd dashboard && npm install && npm run dev
4. Code integration: Use Tracer and HybridLLMClient to add tracing functions.

Competitor Comparison: Compared with LangSmith, it is fully open-source and self-hosted; compared with Langfuse, it has finer cost tracking granularity and more complete features; compared with Phoenix, it provides budget alerts and batch ingestion API. The hybrid client design is its unique advantage.

AgentTrace is applicable to multiple scenarios:

• R&D debugging: Quickly locate anomalies through tracing and playback;
• Cost optimization: Identify high-consumption links and optimize call strategies;
• Production monitoring: Real-time logs and budget alerts ensure stable operation;
• Compliance auditing: Complete execution records support interpretability and compliance.

As an open-source AI agent observability tool, AgentTrace solves core pain points such as debugging and cost management. Its simple integration method and flexible deployment options are worth developers' attention. With the popularization of AI agents, the demand for observability will continue to grow. AgentTrace's open-source nature and community support make it a potential key infrastructure in this field. It is recommended that developers building agent applications consider including it in their tech stack.