agent-replay: A Terminal Debugging and Replay Tool for Agent Workflows

agent-replay: A Terminal Debugging and Replay Tool for Agent Workflows

agent-replay is a Terminal User Interface (TUI)-based debugging tool developed by Chopin998. It supports step-by-step replay and debugging of non-deterministic agent workflow execution traces, helping developers understand the agent decision-making process and locate issues. The project is open-sourced on GitHub (link), with the latest update at 2026-05-31T17:13:20Z.

Core Value: Solve the non-determinism problem in agent debugging, and provide visibility into the execution process and interactive exploration capabilities.

Project Background and Challenges

With the rapid development of Large Language Models (LLMs), agent-based automated workflows are becoming increasingly popular, but autonomy brings debugging difficulties:

• Agent workflows are non-deterministic; the same input may produce different execution paths, making traditional breakpoint debugging ineffective.
• Developers face difficulties such as inability to reproduce issues, difficulty understanding decision logic, tracking tool call chains, and lack of visibility into the execution process.

agent-replay was created to address these pain points.

Core Features

agent-replay provides three key capabilities:

Trace Parsing and Import

Supports parsing standard JSON-format execution trace logs (including execution status, thinking process, tool calls and parameters, results, timestamps, etc.). After import, the complete execution process can be reproduced locally.

Step-by-Step Execution View

Provides a time-travel debugging experience similar to a video player:

• Step forward/backward browsing
• Jump to a specific time point
• Pause at any step for inspection
• Quickly locate tool calls or decision nodes

Detailed Inspection View

Displays fine-grained information for each step:

• Prompt context
• Original model output
• Reasoning chain (if supported)
• Tool call details (name, parameters, return value)

Technical Implementation

agent-replay is developed in Python, with core modules including:

• app.py: Main application entry, handling interaction and interface rendering
• parser.py: Parses JSON trace files and extracts execution steps
• mock_data.json: Sample data showing the trace format

TUI Design Advantages

Choosing a terminal interface over a graphical interface has the following benefits:

• Lightweight: No need for a graphical environment; can be used on SSH remote servers
• Fast startup: No graphical rendering overhead
• Friendly integration: Easy to integrate into command-line workflows
• Resource-friendly: Suitable for resource-constrained environments

Usage Scenarios and Integration

Applicable Scenarios

• Problem Diagnosis: Replay traces to find specific steps of unexpected behavior
• Behavior Understanding: Analyze agent task decomposition, tool selection, and intermediate result processing
• Regression Testing: Save and compare traces to verify behavior stability
• Team Collaboration: Share traces as error reports to reproduce problem scenarios

Compatibility

Supports frameworks that output standard JSON traces:

• LangChain
• LlamaIndex
• AutoGPT
• Custom agent implementations

The parser.py can be extended to support custom trace formats.

Comparison and Project Significance

Comparison with Traditional Tools

Feature	Traditional Log Viewing	agent-replay
Structured Display	Text search	Interactive step browsing
Context Understanding	Manual association	Automatic association of prompts and outputs
Time Travel	None	Supports forward/backward/jump
Visualization	Plain text	TUI interface with clear hierarchy
Usability	Requires familiarity with log format	Intuitive keyboard operations

Project Significance

• Fills the gap in the agent development toolchain, providing much-needed observability tools
• Open-source model promotes community collaboration and supports more frameworks and formats

Usage Recommendations and Conclusion

Usage Recommendations

1. Ensure Trace Completeness: Agent frameworks need to record complete execution information (prompts, responses, tool calls, etc.)
2. Pay Attention to Privacy and Security: Traces may contain sensitive information; be cautious about storage and sharing security
3. Combine with Log Analysis: agent-replay is suitable for interactive exploration; batch analysis needs to be paired with log tools

Conclusion

agent-replay transforms abstract execution processes into interactive visual traces, lowering the threshold for agent debugging. In today's era of rapid agent technology development, such tools mark the maturity of the agent development ecosystem.