HarchOS: A Set of Cloud-Native Training and Inference Templates for Carbon-Aware AI Workloads

HarchOS is an open-source project by HarchCorp (released on GitHub on May 31, 2026) that provides a complete set of starter templates for carbon-aware AI workloads. It supports PyTorch training, LLM inference, multi-region GPU scheduling, and covers the full lifecycle of AI workloads. The core goal is to help developers optimize performance while reducing carbon emissions, addressing the environmental impact of AI computing.

AI's rapid development leads to huge computing demands, with large language model training producing carbon emissions equivalent to dozens of cars' lifetime emissions. Traditional scheduling focuses on performance and cost but ignores carbon intensity differences across time/regions. Carbon-aware computing aims to optimize emissions via smart scheduling, and HarchOS was born in this context to provide practical templates for carbon-aware AI workloads.

HarchOS uses a modular architecture with key components:

• PyTorch training templates: Distributed training with carbon-aware scheduling (choosing optimal time/region based on real-time grid carbon intensity).
• LLM inference templates: Multi-region deployment and load balancing for 24/7 services to optimize carbon footprint.
• Multi-region GPU training: Dynamic migration across regions to select lowest carbon intensity areas.
• Multi-hub architecture: Coordination and synchronization across data centers.
• Data pipelines: Carbon-aware data preprocessing/ETL.
• Monitoring & pricing: Tools for carbon emission tracking and cost analysis.

HarchOS integrates real-time grid carbon intensity data (gCO2/kWh) via APIs. Its scheduling decisions include:

1. Temporal shifting: Delay non-urgent tasks to low carbon intensity periods (e.g., solar peak).
2. Geographic shifting: Migrate workloads to regions with lower current carbon intensity (balancing data transfer cost and carbon benefits).
3. Capacity scaling: Increase resources during low carbon periods, reduce during high (follow-the-sun strategy). Continuous optimization via monitoring module: Track actual emissions and feedback to improve future scheduling.

HarchOS templates apply to various scenarios:

• Large model training: Choose lowest carbon intensity 72-hour window for flexible tasks.
• Inference services: Route traffic to low carbon regions when local intensity rises.
• Batch processing: Data preprocessing/model evaluation (high scheduling flexibility).
• Dev environments: Reduce unnecessary emissions and foster eco-awareness.

HarchOS reflects AI industry's focus on sustainability:

• Environmental responsibility: Helps reduce AI's carbon footprint.
• Cost optimization: Clean energy often has lower prices, so carbon-aware scheduling cuts both emissions and costs.
• Compliance & brand: Supports ESG goals and meets carbon regulations.
• Tech demonstration: Open-source project shows feasibility of carbon-aware AI, promoting wider adoption.

HarchOS lowers the barrier to green AI by providing carbon-aware templates. Future prospects: As carbon data accuracy improves and multi-region cloud infra matures, carbon-aware AI may become standard. HarchOS will play a key role in this transition. Developers are encouraged to adopt tools like HarchOS to balance AI progress and sustainability.