OpenEnv Email Classification System: An Intelligent Customer Service Agent Combining LLM and Reinforcement Learning

openenv-email-triage-rl is an email classification environment compliant with OpenEnv specifications. It integrates large language model (LLM) semantic understanding and Q-learning reinforcement learning to realize automated email processing decisions, addressing the problems of high manual classification costs, response delays, and the inability of rule-based systems to handle complex content in traditional customer service emails.

Customer service mailboxes receive a large number of inquiries, complaints, and requests every day. Manual classification and response are costly, and delays affect satisfaction. Traditional rule-based systems struggle to handle complex and ambiguous content; while LLMs have improved text understanding capabilities, relying solely on them has issues such as high costs, large delays, and difficulty in optimization, requiring an efficient decision-making mechanism.

openenv-email-triage-rl simulates the scenario of an AI agent handling emails. The agent needs to decide to reply directly, escalate for processing, archive, or request supplementary information. Its uniqueness lies in combining LLM semantic understanding and Q-learning reinforcement learning—utilizing the general understanding ability of LLMs while achieving decision optimization and cost control through reinforcement learning.

OpenEnv Compliance Design

Follows OpenEnv specifications and implements standard interfaces like reset(), step(), and state(), facilitating integration with reinforcement learning toolchains and reproducible testing.

Typed Data Model

Uses Pydantic modeling to ensure type safety and data validation, with clear schemas improving maintainability and API integration efficiency.

FastAPI Server

Provides asynchronous endpoints and automatic OpenAPI documentation, making it easy to integrate into existing customer service workflows.

Deterministic Scoring System

Produces the same output for the same input, ensuring result reproducibility and facilitating benchmark testing and debugging.

Task Difficulty Grading

Divides tasks into three levels: simple (e.g., working hours inquiry), medium (refund/bill), and difficult (system failure), adapting to complexity and refining performance evaluation.

Reward Function Design

Correct action: +1.0, partially correct: +0.5, wrong: 0.0, step penalty: -0.1 × number of steps—encouraging efficient and optimized decisions.

Q-learning Optimization

The agent updates action values through interaction, converging to the optimal policy. After training, local inference reduces latency and API costs.

Configure LLM connections via environment variables: API_BASE_URL (endpoint address), MODEL_NAME (model name), HF_TOKEN (Hugging Face token), supporting access to OpenAI API, open-source models, and private deployments.

Provides enterprises with a scalable and optimizable email processing solution. Compared to rule engines, it can handle complex language; compared to pure LLM solutions, it improves cost-effectiveness and response speed. The system is learnable—its decision-making ability improves and error rate decreases as processing volume increases.

This system demonstrates the innovative integration of AI technologies. The architecture combining LLM understanding and reinforcement learning decision-making has reference significance for intelligent decision-making scenarios. The popularization of OpenEnv standardized interfaces will promote more hybrid AI systems from the laboratory to production.