# Zero Trust SRE Gym: A New Reinforcement Learning Environment for Training AI Agents to Operate Compliantly in Zero Trust Security Architecture

> Zero Trust SRE Gym is a reinforcement learning environment built on OpenEnv, which encodes enterprise compliance requirements as hard constraints via Constrained Markov Decision Process (CMDP) to train AI security agents to follow approval processes and avoid unauthorized operations in zero-trust networks. It was selected into the Top 100 at the Meta PyTorch OpenEnv Hackathon 2026.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-11T16:14:29.000Z
- 最近活动: 2026-05-11T16:19:56.549Z
- 热度: 163.9
- 关键词: 强化学习, 零信任安全, AI代理, 网络安全, CMDP, 合规性, OpenEnv, LLM, SRE, 数字孪生
- 页面链接: https://www.zingnex.cn/en/forum/thread/zero-trust-sre-gym-ai
- Canonical: https://www.zingnex.cn/forum/thread/zero-trust-sre-gym-ai
- Markdown 来源: floors_fallback

---

## Zero Trust SRE Gym: A New RL Environment for Compliant AI Agents in Zero Trust Architecture

Zero Trust SRE Gym is an OpenEnv-based reinforcement learning (RL) environment that encodes enterprise compliance requirements as hard constraints via Constrained Markov Decision Process (CMDP). It trains AI security agents to follow approval processes and avoid unauthorized operations in zero trust networks. The project was selected into the Top 100 at the Meta PyTorch OpenEnv Hackathon 2026. Key features include real-time microservice digital twin, adaptive threat generation, and LLM-driven approval evaluation.

## Project Background & Core Problem

Modern SOCs face a huge workload gap—human analysts handle only ~10 events per shift vs thousands daily. Autonomous AI agents are expected to bridge this gap, but zero trust architecture requires explicit authorization for any action. Existing RL security environments prioritize speed over compliance, leading to potential violations (audits, penalties, outages). This raises the "bounded autonomy" problem: balancing AI speed with human governance. Zero Trust SRE Gym addresses this by framing compliance as CMDP constraints.

## Core Design Principles & Methodology

The gym shifts from "doing the right thing" to "doing the right thing the right way" by using architectural constraints instead of just rewards. Unauthorized actions (e.g., isolating nodes without approved tickets) result in immediate -20 penalty and round termination. It models compliance as CMDP: state space (system status, alerts, ticket/approval status), action space (investigate, submit ticket, wait for approval, isolate), reward function (task completion + compliance), and hard preconditions (isolation requires approved ticket).

## Technical Architecture Details

Key components:
1. **Real-time Microservice Digital Twin**: 3 Flask services (frontend, payment, HR DB) with persistent, causal states. Service breaches generate real SIEM logs, and failures propagate via dependency graphs (e.g., isolating frontend affects HR DB).
2. **Adaptive Threat Generator**: Uses real CVE data (score ≥7.5) + Groq LLM to synthesize unique SIEM logs, evolving with agent capabilities.
3. **LLM-driven Approval Evaluator**: Randomly assigns roles (junior SRE, senior SRE, CISO) to evaluate ticket justifications, requiring forensic-quality docs for approval.

## Training Results & Evidence

Trained with GRPO, the agent showed significant improvements:
| Metric | Untrained | Trained |
|--------|-----------|---------|
| Average Reward | -13.8 | +9.7 |
| Policy Compliance | Near 0 | ≥0.82 |
Trained agents learned that following processes is the only path to positive rewards.

## Project Significance & Industry Value

Selected into Top100 at 2026 Meta Hackathon (31k+ teams,71k+ participants). It solves real industry pain points: AI speed vs governance, autonomy vs approval, efficiency vs compliance. It sets new standards for AI agent training by integrating compliance into the environment. Tech contributions: first zero-trust RL environment, CMDP formalization of compliance, real-time digital twin, adaptive learning, LLM-driven evaluation.

## Usage & Extension Options

Resources available: HuggingFace Spaces (online experience), Colab notebooks (training code/tutorials), HuggingFace blog (tech解读). For customization: add new microservices, define new compliance constraints, integrate other LLM backends, modify threat generation strategies.

## Conclusion & Future Impact

Zero Trust SRE Gym represents a new AI training paradigm—teaching agents to act correctly under complex constraints. It translates abstract governance into concrete training signals, emphasizing "doing the right thing the right way". As AI is used in critical infrastructure, this bounded autonomy approach will be key to safe, compliant AI operations, setting a technical benchmark for the field.