AgentGym: An Open-Source Framework for Self-Evolving AI Agents in Diverse Environments

AgentGym is an open-source framework for developing and evaluating general LLM-based agents. It supports 14 diverse interaction environments, provides a unified ReAct format interface, and includes the high-quality trajectory dataset AgentTraj and evaluation benchmark AgentEval. Additionally, the team released AgentGym-RL in September 2025, an extension enabling reinforcement learning for long-horizon decision-making tasks.

Building generalist agents capable of handling diverse tasks and self-evolving across environments is a long-term AI goal. However, existing methods have two key limitations:

1. Imitation learning relies on manual supervision, requiring large labeled data and limiting autonomous exploration.
2. Isolated training in single environments leads to 'expert' agents with poor cross-environment generalization. AgentGym aims to address these issues by enabling the development of self-evolving general LLM agents.

AgentGym's core includes three key elements:

1. Diverse Environments: 14 types covering web navigation (WebShop, WebArena), text games (MAZE, Wordle), household tasks (ALFWorld, SciWorld), digital games (BabyAI, TextCraft), tool use (Weather, Movie, Academia, Sheet, TODOList), and programming (BIRD SQL). All use a unified ReAct interface.
2. AgentTraj-L Dataset: Thousands of high-quality trajectories (e.g., 3930 for WebShop, 2420 for ALFWorld) providing foundational knowledge.
3. AgentEvol Method: Enables cross-task/environment self-evolution, with experiments showing performance comparable to state-of-the-art models.

AgentGym uses a distributed service architecture:

• Standard API: Each environment offers uniform interfaces: /createEnv (create instance), /observation (get state), /available_actions (list actions), /step (execute action), /reset (reset environment).
• Core Components:
  • EnvServer: Hosts environments and provides services.
  • EnvClient: Encapsulates server services into callable functions.
  • AgentController: Connects agents to environments for evaluation, data collection, and training. This design decouples environments from core logic, ensuring scalability.

AgentGym provides the AgentEval benchmark covering 14 environments for standardized evaluation. Key open-source resources on Hugging Face:

• AgentGym/AgentEval: Evaluation dataset.
• AgentGym/AgentTraj-L: Large-scale trajectory dataset.
• AgentGym/AgentEvol-7B: Pre-trained model weights. These resources enable fair comparison between different agent methods.

Released in September 2025, AgentGym-RL introduces reinforcement learning for LLM agents:

• Supports multi-turn RL for long-horizon decision-making tasks.
• Enables large-scale parallel execution (e.g., in WebArena).
• Includes a visualization frontend for trajectory replay and step-by-step analysis. The related paper AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning is also available.

AgentGym's open-source nature brings significant value:

• Lower Research Threshold: Unified interfaces, pre-trained models, and benchmarks reduce infrastructure setup time.
• Standardized Comparison: AgentEval allows fair evaluation of different methods.
• Self-Evolution Support: AgentEvol demonstrates the potential for agents to exceed training data limits.
• Scalable Ecosystem: Modular design encourages community contributions (e.g., new environments like robotics or multi-agent collaboration). Future prospects include more diverse environments, improved autonomous agents, and continued community growth.