Complete Guide: Self-Hosted Large Language Model Lab on AWS

This article introduces how to use Terraform and GitHub Actions to automatically deploy an LLM inference environment on AWS EC2 g5 instances. Core components include the Ollama inference service, Open WebUI chat interface, and multi-engine TTS speech synthesis system. This project is maintained by carlosacchi (GitHub link: https://github.com/carlosacchi/self-hosted-llm-guide) and aims to solve technical barriers to self-hosting LLMs, enabling data privacy protection, cost control, and full control over the inference process.

With the development of LLM technology, developers and enterprises want to deploy models in private environments. Core motivations include data privacy, cost control, and control over the inference process. However, self-hosting faces barriers such as GPU driver installation, CUDA configuration, model downloading, and UI development. This project provides an automated solution to help users quickly build a production-ready LLM inference environment.

Core Components: Ollama inference engine (Dockerized, GPU-accelerated), Open WebUI (port 3000), multi-engine TTS (Kokoro, XTTS-v2, Piper), Gradio TTS management interface (port 7860). Network Architecture: AWS VPC (10.42.0.0/16), public subnet, internet gateway, route table, security groups to restrict inbound traffic. Computing Resources: Based on AWS Deep Learning AMI (Ubuntu 22.04), supports g5 series instances (single GPU like g5.xlarge, multi-GPU like g5.12xlarge). Access Control: Security groups only allow specified IPs to access ports 22 (SSH optional), 3000 (WebUI), 7860 (TTS), and 8000 (FastAPI).

Preparation Phase: Need to prepare AWS credentials (EC2/VPC/IAM/S3/EventBridge permissions), S3 bucket (Terraform state), public IP, and EC2 key pair (optional). GitHub Actions Deployment: Configure repository secrets (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, TF_STATE_BUCKET), supports apply/destroy operations, and allows selection of instance specifications, regions, and other parameters. Local Terraform Deployment: Initialization command (terraform init) and application command (terraform apply), which require specifying variables such as bucket, key, region, instance_type, and ipv4_allowed.

TTS Engines: 1. Kokoro (9 voices, optional GPU, low latency); 2. XTTS-v2 (21 voices + voice cloning, GPU required, high quality); 3. Piper (multilingual, CPU-only, lightweight). Cost Optimization: EventBridge Scheduler automatically stops instances every night (note that EBS/elastic IP still incur charges); region selection recommendations (eu-west-1 has high cost-effectiveness, us-east-2 has low prices).

Application Scenarios: Enterprise private AI deployment (data-sensitive scenarios), developer experimental environment (no API limits), model fine-tuning (multi-GPU support), voice application prototype development (TTS comparison). Post-Deployment Access: Terraform outputs instance ID/IP, Open WebUI address (http://:3000), Gradio TTS address (http://:7860), and SSH command (if the key is configured).

This project provides a clear path for private LLM deployment, enabling automated and repeatable deployment via Terraform and GitHub Actions. Its value lies in demonstrating the engineering methodology for AI infrastructure. It is recommended that users start with g5.xlarge and pay attention to updates of Ollama and Open WebUI. In the future, such automated solutions will become an important part of enterprise AI strategies.