CompanyOS：训练AI代理应对真实企业混乱的强化学习环境

CompanyOS is a reinforcement learning environment developed for the Meta PyTorch OpenEnv Hackathon, simulating the chaos and complexity of real enterprise systems. It enables AI agents to learn multi-step workflows across three interconnected applications (TicketDesk, DataHub, ApprovalFlow). Unlike existing benchmarks, it focuses on testing agents in complete, messy workflows similar to real-world scenarios, aiming to be a controlled, reproducible benchmark for enterprise reasoning.

Enterprise AI agents often fail not due to lack of intelligence but because real systems are chaotic, inconsistent, and interdependent. Examples include tickets with missing priorities, expired compliance data, OOO approvers, random approval rejections, and irreversible actions with penalties. Existing RL benchmarks test agents on clean, isolated tasks, which don't reflect real enterprise chaos—this gap is what CompanyOS addresses.

CompanyOS is an OpenEnv-compliant RL environment with three core apps:

• TicketDesk: Simulates Jira-like ticket management with intentional chaos (missing priority/status/assignee, blocked tickets).
• DataHub: Stores business metrics with stale data (e.g., expired vendor compliance scores) requiring refresh.
• ApprovalFlow: Handles approvals with OOO approvers, random rejections, and required field checks. Its design philosophy solves real system issues: clean resets between episodes, no rate limits, adjustable chaos levels, and rewards causal reasoning over pattern matching.

A typical vendor onboarding task involves steps like setting ticket priority, verifying tickets, refreshing stale data, routing to OOO approver delegates, and submitting approvals. The reward system uses shaped rewards (e.g., +1.0 for setting priority, +1.5 for verifying tickets) plus a large termination reward (15.0 for success). Random agents average -2.04 with ~2% success rate due to incorrect tool order or ignoring warnings.

Tech stack includes OpenEnv-core (RL framework), FastAPI (API server), Docker (containerization), HuggingFace Spaces (hosting), Unsloth (LLM fine-tuning), and Qwen2.5-1.5B-Instruct (base model). Usage is simple: 5 lines of code to reset the environment and execute actions. Key endpoints include /reset, /step, /render, etc.

CompanyOS stands out from benchmarks like WebArena (single system), WorkArena (vendor-locked), AgentBench (no partial observability), ToolBench (no state), and ALFWorld (non-enterprise). Its unique value: multi-system causal reasoning, data quality as a core challenge, and being open/reproducible (free public REST API).

Training uses Unsloth + HF TRL for GRPO optimization. Progress has three stages: stopping invalid tool calls, learning correct app routing for subtasks, and mastering full workflows (including stale data and OOO routing). Random baselines are available to measure pre-training performance.

Current limitations: uses small Qwen2.5-1.5B model (may limit complex reasoning). Future directions: larger models, multi-modal observations (screenshots), more complex scenarios (cross-department collaboration), and integration with other RL frameworks. Conclusion: CompanyOS is an innovative benchmark for training robust AI agents that can handle real enterprise chaos, providing value for teams developing enterprise AI solutions.