AWS Bedrock SaaS Platform: Multi-tenant LLM Inference Service Architecture for EU Enterprises

The open-source project saas-bedrock-platform is a B2B SaaS platform for EU enterprises. Its core goal is to help service providers resell AWS Bedrock large model inference services in compliance with regulations. The platform uses a multi-tenant architecture, integrates AWS Organizations and LiteLLM Gateway, addresses pain points such as GDPR compliance, resource isolation, and cost sharing, and provides an efficient Bedrock service distribution layer for the EU market.

With the popularization of LLMs in enterprise scenarios, B2B service providers face three major pain points when integrating AI capabilities: strict GDPR data sovereignty requirements, complex multi-tenant resource isolation, and difficulties in cost sharing and billing management. The saas-bedrock-platform open-source project emerged to provide a complete solution to help enterprises quickly build a Bedrock resale platform for the EU market.

Multi-tenant Isolation Mechanism

Uses AWS Organizations, where each customer corresponds to an independent account/organizational unit, achieving account-level resource isolation, avoiding data leakage and resource contention, and supporting hierarchical operations.

LiteLLM Gateway Integration

Unified OpenAI-compatible API interface, supporting Bedrock models like Claude and Llama; intelligent routing to select the optimal endpoint; request-level usage tracking provides data for billing.

GDPR Compliance Design

Inference requests are routed to AWS European regions (e.g., eu-west-1) to ensure data residency; complete access logs and audit trails; logical isolation and encrypted storage of tenant data.

Cost Sharing & Metering

Built-in refined usage metering system that counts token consumption by tenant, model, and time period, supporting flexible pricing and subscription package design.

Scalable Deployment Model

Supports deployment from single-account verification to multi-organization production, using AWS CloudFormation/Terraform for Infrastructure as Code (IaC) to ensure environment consistency.

Developer-Friendly Access

Provides clear API documentation and SDK examples; downstream developers can integrate Bedrock capabilities with just a few lines of code, reducing access friction.

The project lowers the technical threshold for the EU AI service market and is suitable for three types of organizations:

1. Traditional software companies: Quickly obtain compliant LLM inference backends;
2. Vertical AI startups: Focus on model tuning and application innovation;
3. Cross-regional AI service providers: Quickly launch in the EU market.

1. Relies on the AWS ecosystem; non-AWS users need to evaluate migration costs;
2. GDPR compliance requires supporting legal texts and data agreements;
3. The multi-tenant account model increases AWS account management complexity, requiring the establishment of operation and maintenance processes.

The saas-bedrock-platform focuses on solving enterprise-level deployment and compliance pain points without rebuilding underlying models. With the implementation of the EU AI Act, such compliance infrastructure will become more important. The project's modular design supports customized expansion and serves as an important reference implementation for AI service providers in the EU market.