Building a 4-Machine MLOps Home Lab: A Complete Practice from Data Pipeline to Local Inference

This article details the construction plan of a 4-machine MLOps home lab, covering the layered architecture of storage, computing, GPU inference, and control plane, VLAN network design, MLOps workflows, and end-to-end machine learning deployment practices. This lab provides practitioners and enthusiasts with full control, predictable costs, unrestricted experimental freedom, and in-depth understanding of underlying technologies. It serves as both a practical work environment and a learning project and skill showcase platform.

In an era dominated by cloud computing, local MLOps labs still have unique value: full control, predictable costs, unrestricted experimental freedom, and in-depth understanding of underlying technologies. This 4-machine lab project demonstrates an end-to-end machine learning platform, covering everything from data storage to model training, workflow orchestration to local inference, and serves practical, learning, and skill showcase functions.

4-machine layered architecture: 1. Antsle Node (Storage Layer): TrueNAS + ZFS provides reliable distributed storage with support for snapshots, compression, and deduplication; 2. Mac Pro Node (Data and Orchestration Layer): PostgreSQL, MinIO, Apache Airflow, Jupyter, responsible for data management, task scheduling, and development; 3. MSI Node (GPU Computing Layer): GPU supports LLM inference, training, and fine-tuning; 4. MacBook Node (Control Plane): Management entry and development workstation. Network design: Use Cisco switches + Palo Alto firewalls to implement VLAN segmentation (management/storage/computing/external access networks) to achieve security isolation, traffic optimization, and fault domain limitation.

Storage Layer: TrueNAS is based on ZFS, with core features including data integrity (checksum + automatic repair), snapshots (version control/rollback), and compression/deduplication (space saving); Data Orchestration Layer: PostgreSQL stores metadata, MinIO provides S3-compatible storage, Airflow orchestrates workflows (DAG handles dependencies and scheduling), Jupyter supports interactive development; GPU Layer: Local LLM inference solutions (Ollama/vLLM/Llama.cpp), model quantization (FP16 → INT8/INT4 to reduce memory usage), inference serviceization (OpenAI-compatible API).

Complete workflow: 1. Data Ingestion: Raw data enters the Antsle storage layer (automated by Airflow); 2. Preprocessing: After exploration in Jupyter, convert to Airflow tasks and output to MinIO; 3. Feature Engineering: Transform features into feature storage; 4. Training: MSI node uses distributed frameworks for training, with metrics/checkpoints recorded to MLflow;5. Evaluation: Evaluate using validation sets;6. Deployment: Convert models to inference services (triggered by Airflow/CI/CD);7. Monitoring: Continuously monitor performance and retrain if necessary.

Phased implementation: Infrastructure preparation → Network configuration → Storage deployment → Computing layer setup → Service deployment → GPU environment configuration → Workflow development → Documentation maintenance. Learning value: System management, containerization, MLOps practice, network security, and troubleshooting skills. Cost-effectiveness: One-time hardware investment (long-term amortization lower than cloud services), power and maintenance costs; learning benefits and full control (no cloud restrictions/privacy concerns).

Conclusion: This 4-machine lab shrinks enterprise-level MLOps architecture into a home environment, with practical (production-level workflow), learning (theory to practice), and exploration (technology playground) values. It is a way to prove technical capabilities and deeply understand the essence of technology. Future expansion: Kubernetes integration, more GPU nodes, edge inference, multi-cloud hybrid, and IaC implementation with Ansible/Terraform.