Servixa: A Structured Evaluation Environment for AI Customer Service Systems

Servixa is a structured simulation environment built on the OpenEnv framework, focusing on evaluating the operational decision-making capabilities of AI agents in customer service scenarios (such as risk identification, priority setting, ticket routing, response selection, safe closure, etc.) rather than just text generation quality. It provides reproducible benchmark tests, connects to real customer service workflows, and helps developers and researchers improve the practical decision-making capabilities of AI systems.

Traditional AI customer service evaluations mostly focus on the quality of response text, ignoring the complex operational decisions (classification, assignment, risk judgment, etc.) of real customer service teams. Servixa is designed around the core question "Can the agent make correct support decisions?", requiring agents to have capabilities such as risk identification, priority setting, routing assignment, response selection, and safe closure, which are close to real production environments.

Servixa adopts a four-layer architecture: FastAPI application layer (providing OpenEnv-compatible interfaces), core environment (coordinating evaluation processes), task definition (including scenarios, ticket lists, hidden expectations), and deterministic scorer (scoring from six dimensions: classification (20%), priority (15%), routing (20%), template (15%), solution (20%), and closure safety (10%)). Tasks are divided into three difficulty levels: easy (password reset, logistics delay), medium (duplicate order, abuse report), and hard (account breach, VIP service interruption).

Servixa provides a reproducible baseline implementation. Local test scores: easy task 1.0, medium 0.95, hard 0.9625, average 0.9708. Deployment supports local operation (uvicorn startup, Docker container), Hugging Face Spaces (based on Docker configuration), and provides a submission script (inference.py) for interaction with models.

Servixa is suitable for scenarios such as benchmarking LLM agents, comparing strategies, evaluating safe routing behaviors, and researching reward shaping. Its advantages include real domain scenarios, deterministic scoring, meaningful reward shaping, clear interfaces, strong baselines, and real-time deployment capabilities.

Potential improvement directions for the project: adding environment loop GIFs/videos, providing weaker comparison baselines, and writing instructions for customer service routing as an RL benchmark. Summary: Servixa represents an important direction for AI evaluation from text generation to operational decision-making, providing a practical and strict benchmark platform for AI customer service systems, helping to improve the practical decision-making capabilities of agents.