Guardian-Mesh: Enterprise-Grade AI Governance Control Plane

Guardian-Mesh is an open-source enterprise AI gateway that builds a governance layer between users and large language models (LLMs). It addresses critical enterprise AI governance gaps by enabling request interception, security policy execution, cost optimization, and compliance auditing. The solution supports multi-cloud environments and local deployment, filling the void left by most existing solutions that focus on RAG architecture and feature development but ignore the essential governance layer in production.

The rapid adoption of generative AI in enterprises brings opportunities but also exposes severe governance gaps:

• Cost out of control: Uncontrolled usage and lack of intelligent routing lead to exponential API bill growth.
• Data leakage risks: Employees may accidentally leak sensitive data like customer PII, internal credentials, or trade secrets in prompts.
• Compliance blind spots: Lack of complete AI interaction logs makes it hard to meet GDPR, HIPAA, etc.
• Security vulnerabilities: New threats like prompt injection and jailbreak attacks lack effective protection. Most solutions overlook the governance layer, which Guardian-Mesh aims to fill.

Guardian-Mesh uses a control plane architecture, inserting a high-performance governance layer between users and LLMs. Its core idea: security policies should be executed before model calls, not relying on the model's own security training.

Request flow: User → Identity Layer → Governance Grid → Policy Engine → Model Routing → LLM → Response Validation → Audit Ledger

Key advantages:

• Request interception: All AI requests pass through the gateway for policy execution.
• Programmable policies: OPA-style framework supports flexible allow/block/route decisions.
• Identity awareness: Each request is linked to user identity for fine-grained access control and audit.
• Multi-cloud unification: Supports Azure OpenAI, AWS Bedrock, GCP Vertex AI, and local Ollama models.

Security & Compliance Layer

• Local PII detection and desensitization: Identifies and masks sensitive info (emails, credentials) before data leaves the enterprise network (key for data sovereignty).
• Prompt injection protection: Detects and blocks jailbreak patterns and injection attacks.
• Input purification: Cleans prompts to remove potential dangerous content.

Cost Optimization Engine

• Semantic cache: Eliminates redundant API calls via encrypted semantic caching (returns cached results for similar queries).
• Dynamic model routing: Chooses models based on cost, latency, and policies (e.g., simple queries to low-cost models).
• Budget-aware policies: Sets inference budget limits; auto-degrades or blocks when exceeded.

Observability & Audit

• End-to-end request tracking: Full visibility from user request to model response.
• Prompt/response logs: Records all interactions for post-audit and analysis.
• Anomaly & hallucination detection: Identifies abnormal outputs and potential hallucinations.
• Compliance-ready logs: Aligns with GDPR/HIPAA for regulatory reviews.

Guardian-Mesh uses a pragmatic tech stack:

• Frontend: Streamlit-based executive dashboard showing governance status, cost metrics, and audit logs.
• Backend: Python modular governance layer for easy extension and maintenance.
• Storage: SQLite for audit logs and semantic cache (lightweight yet functional).
• Inference: Ollama supports local edge inference; seamlessly integrates with cloud provider APIs.

Deployment is simple: Clone the repo, install dependencies, and run Streamlit—ideal for MVP validation of AI governance concepts.

Guardian-Mesh’s core design principle is data sovereignty first. Traditional cloud-native AI solutions require data to be sent to cloud providers, which is unacceptable for sensitive industries (finance, medical, government).

Solutions:

1. Local PII desensitization: Sensitive info is identified and masked before leaving the enterprise network.
2. Local inference option: Full offline inference via Ollama.
3. Encrypted cache: Semantic cache uses encrypted storage to avoid plaintext exposure.

This design allows enterprises to use LLM capabilities while maintaining full control over data.

Enterprise Expansion Roadmap

Current version is an MVP; future plans include:

• Azure Entra ID integration (connect to existing enterprise identity systems).
• Advanced NER models (more accurate PII detection).
• Policy-as-Code engine (declarative policy configuration).
• Distributed logs & monitoring (centralized observability for large-scale deployments).

Application Scenarios

Guardian-Mesh is ideal for:

• AI governance pilots: Enterprises wanting to validate governance concepts without heavy investment.
• Multi-cloud environments: Unifying governance across multiple cloud LLMs.
• Compliance-sensitive industries: Finance, medical, government (strict data sovereignty and audit requirements).
• Cost-sensitive scenarios: Reducing AI operational costs via caching and intelligent routing.

Its open-source nature allows customization and avoids vendor lock-in.