SmartMail Env: A Reinforcement Learning Environment for Email Classification and Customer Support Workflows

SmartMail Env is a reinforcement learning environment built on the OpenEnv standard, designed to train and evaluate AI agents' ability to handle email classification and end-to-end customer support workflows, covering real-world scenarios like refunds, delivery delays, and payment failures. It fills the gap between pure dialogue quality testing and practical business process decision evaluation, providing a standardized testing platform for the practical application of AI agents.

SmartMail Env originated from the Meta×Hugging Face OpenEnv Hackathon project, with the core goal of simulating an RL environment for real customer support scenarios. Unlike benchmarks that only focus on dialogue quality, it emphasizes the agent's decision-making ability throughout the entire workflow from receiving emails to resolving issues, reflecting the trend of AI applications shifting from language generation to practical task execution capabilities.

SmartMail follows the OpenEnv standard interface, with an interaction model of Agent→Action→Environment→Reward→Next State, compatible with mainstream RL frameworks.

• Observation Space: Includes email subject, body, and current status (e.g., new/under_review/resolved), simulating the working mode of real customer support systems.
• Action Space: Supports four core actions: classify (categorize), escalate (transfer to human), mark_spam (mark as spam), and resolve (mark as resolved), covering key decision points in customer support.

SmartMail has designed tasks at three difficulty levels:

• 🟢 Easy Level: Refund issues (identify and initiate the process), delivery delays (check logistics updates) — clear and easy-to-execute processes.
• 🟡 Medium Level: Payment failures (diagnose multiple causes), account login issues (troubleshoot) — require detailed analysis and multi-step handling.
• 🔴 Hard Level: Phishing/security escalation (identify threats and trigger protocols), mixed spam and urgent complaints (identify priorities amid noise) — test complex reasoning and high-risk decision-making abilities.

A progressive reward shaping strategy is adopted, with the total reward broken down into:

Reward Type	Score	Description
Correct Action	0.4	Selecting the appropriate action type
Correct Label	0.2	Accurate classification label
Correct State Transition	0.2	Workflow state transition meets expectations
Completion Reward	0.2	Successfully completing the entire process

Example flow: new→under_review (reward 0.6: correct action + state transition) → resolved (reward 0.2: completion), total reward of 0.8, encouraging reasonable multi-step strategies.

The tech stack includes OpenEnv (base framework), FastAPI (API interface), Docker (containerization), Hugging Face Spaces (online demo), and Python 3.12.

• Local run: python inference.py
• Docker deployment: docker build -t smartmail-env . && docker run --rm smartmail-env
• Environment validation: openenv validate Online demo link: https://duniyakapapa007-smartmail-env.hf.space

The value of SmartMail is reflected in:

1. Model Capability Evaluation: Test the performance of different LLMs in real customer support scenarios;
2. RL Training Platform: Train specialized customer support agent strategies;
3. Workflow Optimization: Enterprises test and optimize their own customer support workflows;
4. Security Education: Train agents' security awareness through phishing email identification tasks.

SmartMail Env represents an important direction in the shift of AI evaluation benchmarks from language ability to task execution ability, providing a practical tool for AI agent development. As AI agents become more popular in enterprise applications, such environments will become more important. In the future, more industry-specific OpenEnv environments are expected to emerge, promoting the practical application and standardization of AI agent technology.