CustomerSupportEnv: A Practical Training Ground for AI Customer Service Agents

CustomerSupportEnv is a customer support simulation environment built on OpenEnv, featuring 40 real-scenario tickets. It aims to train and evaluate the comprehensive customer service handling capabilities of AI agents. It addresses the training and evaluation challenges faced by AI customer service as it evolves from simple FAQ answering to complex ticket processing. By simulating the complete customer service workflow (classification, priority judgment, response drafting, resolution/escalation decision-making), it helps agents develop transferable general customer service skills.

Customer service is an indispensable part of business operations but comes with high labor costs. With the development of large language model technology, AI customer service agents are evolving from simple FAQ answering to complex ticket processing. However, systematically training and evaluating their real capabilities has long been an industry challenge. The CustomerSupportEnv project was created to address this issue, providing a simulation environment close to real scenarios, allowing agents to learn and grow in a safe and controlled manner.

CustomerSupportEnv simulates the complete customer support workflow, requiring agents to handle ticket classification, priority determination, response drafting, and resolution/escalation decisions. The environment includes 40 real scenarios covering four major categories:

• Billing Issues: Charge disputes, refund requests, invoice problems
• Logistics Issues: Delivery delays, address changes, lost packages
• Technical Issues: Product malfunctions, usage questions, compatibility problems
• General Inquiries: Product information, account issues, other questions This diversity ensures agents do not overfit to a single domain and cultivates general customer service capabilities.

Agents need to make four key decisions:

1. Issue Classification: Accurately determine the ticket's category (billing/shipping/technical/general) to lay the foundation for subsequent processing;
2. Priority Determination: Assign low/medium/high/urgent priorities based on issue type, customer account level, etc.;
3. Customer Response: Draft natural, professional, and targeted responses that demonstrate empathy;
4. Processing Decision: Decide to resolve or escalate, balancing problem resolution and labor costs. The observation space includes ticket_id, customer name, account level (basic/premium/enterprise), issue description, conversation history, reward score, round status, etc. The account level design simulates VIP priority strategies and fairness challenges.

The scoring mechanism combines rule-based judgment and LLM-as-judge:

Scoring Dimension	Score	Judgment Method
Correct Classification	0.2	Rule Matching
Correct Priority	0.1	Rule Matching
Correct Escalation Decision	0.2	Rule Matching
Response Quality	0.0-0.5	LLM Judgment
Difficulty is divided into three levels:

• Easy (ticket-classify): Only classification and priority determination required;
• Medium (ticket-resolve): Classification + response + resolution/escalation decision;
• Hard (queue-management): Handle a queue of 5 tickets with mixed urgency levels, simulating resource allocation challenges.

Built on the OpenEnv framework, it follows standardized environment interfaces and can seamlessly integrate with RL algorithms and training frameworks. The tech stack includes:

• uv: Fast Python package manager;
• Pydantic: Data validation and serialization;
• FastAPI: Optional server-side API. Deployment methods:
• Local: pip install uv → uv sync → uv run server;
• Docker: docker build -f server/Dockerfile . → docker run -p 8000:8000 <image>. Supports local debugging and integration with cloud training infrastructure.

Application Value:

• Researchers: A standardized benchmark environment in the customer service domain for model comparison and method exploration;
• Developers: A sandbox testing environment for intelligent customer service systems to pre-evaluate capabilities and stress;
• Business Decision-makers: Evaluate the maturity of AI customer service technology to assist investment decisions. Future Expansion:
• Multi-turn dialogue support;
• Customer sentiment analysis and response strategy adjustment;
• Knowledge base integration;
• Multi-language support;
• Real-time response time pressure simulation.