SharedLLM: A Community-Driven Distributed AI Inference Network Turning Idle Computing Power into Public Infrastructure

SharedLLM is an open-source distributed LLM inference network whose core goal is to aggregate idle computing resources from individuals and institutions to build community-owned AI infrastructure. It aims to break the computing power monopoly of a few tech giants, allowing users to run cutting-edge large models by only paying for electricity and bandwidth costs, thus achieving the democratization of AI computing resources.

Currently, a few tech giants control most of the world's AI inference infrastructure, running services at low costs but renting them at high prices, forming a computing power monopoly. Meanwhile, millions of laptops, workstations, school lab and office devices around the world are largely idle during non-working hours (e.g., after 5 PM). If these scattered computing resources are effectively organized, they can become a huge alternative AI resource pool—SharedLLM was born based on this observation.

SharedLLM adopts a federated architecture with core components including:

1. Coordinator: The central hub responsible for node registration, task scheduling, and settlement management, providing FastAPI interfaces and Python SDK, supporting self-hosting or Naridon hosting (does not own computing power);
2. Node Daemon: A lightweight service (supports multiple systems) that registers device capabilities, sends heartbeats, and serves inference shards; nodes can independently control joining/leaving;
3. RPC Layer & Cross-Machine Parallelism: Based on llama.cpp's RPC protocol, with HMAC-SHA256 authentication, supporting cross-machine tensor parallelism to break through single-machine memory limits;
4. LAN Auto-Cluster Discovery: Devices in the same subnet with latency <2ms are automatically merged into virtual nodes, suitable for multi-machine environments like computer rooms/offices;
5. Federation Protocol: Supports interconnection between coordinators of different sites, enabling cross-organization resource federation via HMAC signatures, preserving sovereignty while sharing computing power.

SharedLLM provides deployment solutions for different user groups:

• Individual Users: Install tools to contribute idle computing power in exchange for free inference credits, or run private LLMs locally to get OpenAI-compatible APIs (zero cost, zero telemetry);
• Institutional Users: Initialize a site with two commands (sharedllm site init/join), LAN auto-clustering supports tensor parallelism without per-machine configuration;
• Researchers/Civic Tech Advocates: Federate institutional clusters, build public AI services via cross-coordinator calls with HMAC signatures and trust mechanisms.

SharedLLM protects privacy and security by default:

• Data Sovereignty: User prompts and hardware are controlled by users; sensitive workloads can be restricted to their own LAN;
• Transparent No Backdoors: The CLI only communicates with the coordinator URL provided by the user; all source code is auditable;
• Encrypted Transmission: Node authentication uses HMAC-SHA256, working channels use temporary keys, and coordinators support TLS;
• Standard Encryption: Uses verified standard encryption libraries with no custom schemes.

SharedLLM is in the public Alpha phase with the following feature statuses:

Feature Module	Status
Single-node inference (OpenAI-compatible API)	✅ Stable, in production
Two-machine RPC tensor parallelism	✅ Available
LAN auto-cluster discovery	✅ Phase 1 released
Layered scheduling (layer placement)	✅ Phase 2 released
Cluster head range scanning & benchmarking	✅ Phases 3-4 released
CLI site initialization/joining	✅ Phase5 released
Cross-site federation registry (HMAC)	✅ Phase6 released
Browser worker (WebGPU compute layer)	🔨 In progress (Phases7-8)
On-chain credit ledger (EVM)	📝 Testnet contract drafted
Mainnet/hosted coordinator scaling	🎯 Target Q4 2026

SharedLLM uses the AGPL-3.0 license: its network usage terms are contagious, preventing SaaS loopholes from closing open-source projects. Contributions use the Developer Certificate of Origin (DCO), no CLA required, and contributors retain copyright. Naridon Inc. holds the trademark and operates hosted infrastructure, but does not own the code, cannot relicense it, nor exclude the community; it mainly funds maintenance, runs hosted coordinators, and provides commercial support.

SharedLLM is a technical attempt to counter AI computing power centralization, proving that scattered idle resources can be organized into competitive public services. It provides a toolset for developers and institutions concerned about AI democratization and data sovereignty—schools can turn computer lab machines into virtual GPU clusters, and individuals can contribute computing power to provide free AI services for the community. Long-term vision: AI computing should be a public utility, not a rent paid to a few companies.