AgentLog: An Agent Workflow Auditing and Replay Tool Based on OTel GenAI Semantic Specifications

This article introduces the open-source tool AgentLog, a self-hostable agent workflow auditing tool that adheres to OpenTelemetry GenAI semantic specifications. It provides scanning, replay, and auditing capabilities, supports MCP protocol integration with mainstream AI development environments, and is suitable for AI security governance in CI/CD pipelines. The original author is cognis-digital, and the project is open-sourced on GitHub (link: https://github.com/cognis-digital/agentlog), released on 2026-06-08.

With the development of LLM and agent technologies, enterprises integrating AI agents face governance challenges: black box problem (opaque decision-making process), security risks (access to sensitive data or dangerous operations), compliance requirements (auditing AI behaviors), and difficulty in troubleshooting (lack of debugging tools). Traditional APM tools are designed for deterministic systems and are difficult to adapt to non-deterministic systems like agents, requiring a specialized solution.

Core Features: 1. Workflow scanning (supports CLI commands such as agentlog scan ., can output JSON/SARIF, CI gatekeeping function); 2. Multi-format reports (tables, JSON, SARIF); 3. Native MCP support (compatible with Claude Desktop, Cursor, etc.); 4. Cross-platform and multi-language (Linux/macOS/Windows/Docker, Python/JS/Go/Rust versions).

Technical Architecture: Modular pipeline design: Input → Collector → Rule Engine/Analyzer → Scorer → Report Generator (outputs tables/JSON/SARIF/MCP tools).

AgentLog fully supports the OpenTelemetry GenAI semantic specification, which provides a standardized telemetry model for generative AI applications.

Key Semantic Attributes: gen_ai.system (AI system identifier), gen_ai.request.model (request model), gen_ai.response.model (response model), gen_ai.usage.input/output_tokens (token usage), gen_ai.response.finish_reason (finish reason).

Event Types: gen_ai.content.prompt (user prompt), gen_ai.content.completion (model completion content). Following this standard allows integration with any OTel observability platform, avoiding vendor lock-in.

Application Scenarios: 1. CI/CD integration (e.g., using agentlog scan . --fail-on high in GitHub Actions); 2. AI security governance (identify sensitive data leaks, dangerous operations, unauthorized model access, prompt injection vulnerabilities); 3. Compliance auditing (meet GDPR, finance/healthcare industry requirements); 4. Development debugging (replay execution process to locate issues).

Ecosystem Integration: Supports MCP servers, OpenAI-compatible APIs, and frameworks like LangChain/CrewAI/AutoGen/LlamaIndex.

Competitor Comparison: AgentLog's advantages include self-hostable (no account required), single command with zero configuration, JSON+SARIF support for CI, native MCP, multi-language porting, and open license (COCL).

Deployment Methods: 1. Package manager (pip install cognis-agentlog); 2. Docker (docker run ghcr.io/cognis-digital/agentlog); 3. Platform scripts (dedicated scripts for Linux/macOS/Windows); 4. Cloud platforms (AWS/Azure/GCP/k8s, refer to DEPLOY.md).

Limitations: The current version focuses on static code scanning.

Future Outlook: Runtime monitoring, distributed tracing (multi-agent collaboration), behavior baseline (establish normal baselines from historical data), auto-fix (suggest or automatically apply fixes).

AgentLog is an important advancement in the field of AI observability. By adopting OTel GenAI specifications and MCP protocol, it provides enterprises with a powerful and flexible agent governance tool. As AI agents become core infrastructure, such tools are crucial for ensuring AI systems are secure, trustworthy, and compliant. Teams building or operating AI agent systems are advised to evaluate and adopt AgentLog.