xtrace: A Lightweight Observability Platform for AI and LLM Applications

xtrace is a self-hosted observability backend designed specifically for AI/LLM applications and agent workflows. It provides tracing, observation, and time series metrics collection capabilities to help developers diagnose latency, cost, quality, and failure patterns in production environments. Key features include OpenTelemetry compatibility, Langfuse API compatibility, session-oriented metadata propagation, and flexible metrics system.

With the widespread deployment of LLM and agent workflows in production, developers face challenges in monitoring these complex systems. Traditional observability tools struggle to adapt to LLM-specific needs like multi-turn conversation tracking, fine-grained inference cost measurement, and agent decision path visualization. xtrace was created to fill this gap as a lightweight, self-hosted solution tailored for AI/LLM apps.

xtrace is written in Rust and uses PostgreSQL for data storage. Its core features include:

1. Native OpenTelemetry Protocol (OTLP) support for seamless integration with existing pipelines.
2. Langfuse API compatibility layer to allow existing Langfuse clients to switch without code changes.
3. Session-oriented metadata model (session_id, turn_id, run_id, step_id/type) for tracking complex agent workflows.
4. Flexible time series metrics system with custom names/labels, aggregation functions (avg, max, min, sum, p50/p90/p99), downsampling, and grouping.

Deployment is simple with required environment variables: DATABASE_URL (PostgreSQL connection string) and API_BEARER_TOKEN (auth token). Start with cargo run --release (listens on 127.0.0.1:8742 by default). Production guides include systemd service files and GCP deployment with TLS. Health check endpoints: /healthz (process alive) and /readyz (PostgreSQL reachable). Example data ingestion: use HTTP POST to /v1/metrics/batch with metrics payload (e.g., GPU utilization).

Key technical strengths:

1. Rust implementation ensures memory safety and high performance for handling large telemetry data.
2. Reasonable default configurations (20 QPS rate limit, 40 burst capacity, 20MB max request size) balance system protection and out-of-the-box usability.
3. Official Rust client SDK (xtrace-client) with async API for easy integration, supporting metrics push and complex queries (percentile aggregation, grouping).

xtrace is ideal for:

1. Self-hosted LLM service operations (e.g., Xinference) with Langfuse-compatible observability and full data sovereignty.
2. Multi-agent system debugging: track each agent's thought process and action sequence via session metadata.
3. Cost and performance optimization: identify high-cost inference calls, slow queries, and resource bottlenecks using fine-grained tracing and metrics.

xtrace is a significant advancement in AI/LLM observability, offering better developer experience than general-purpose tools. As LLM apps become more prevalent in production, dedicated tools like xtrace will grow in importance. Its open-source nature and active maintenance (complete docs, example scripts) make it a strong choice for building LLM observability infrastructure.