Luckrig: An Innovative Exploration of a Decentralized Local LLM Inference Sharing Platform

Luckrig is a contribution-based local LLM inference API sharing platform designed to resolve the conflict between individual developers' computing power access bottlenecks and users' idle local computing resources, pioneering a new model of community collaborative AI infrastructure sharing. The platform integrates the concept of "contribution-based access rights", allowing users to exchange contributions for the opportunity to access OpenAI-compatible APIs on others' local devices, breaking the monopoly of centralized cloud services and promoting the flow and sharing of computing resources within the community.

In the current development of LLMs, high-performance GPUs are expensive and in short supply, restricting individual developers; at the same time, many users have idle local computing power. Luckrig transplants the "contribution-based access rights" concept from Hotline Connect to the local LLM field, building a community collaborative inference sharing platform. The project name "luckrig" combines "luck" (fortune/lottery) and "rig" (device/equipment), implying that users can exchange contributions for access rights and have the chance to use excellent nodes in the community, breaking the monopoly of centralized cloud services.

Luckrig adopts a modular distributed architecture, with core components including:

1. Tracker: A central coordination node responsible for node registration, health monitoring, and temporary token issuance based on contribution scores. It does not directly participate in inference and serves as the hub for resource scheduling and service discovery.
2. Node Proxy: A proxy layer deployed in front of ollama or llama.cpp, providing OpenAI-compatible APIs and supporting Plain Mode (standard TLS encryption) and Subtext Mode (encrypted payload embedded via Unicode variant selectors).
3. CLI Tool: Completes node registration and proxy configuration with a single command, lowering the entry barrier for participation.
4. Web Interface: Offers functions such as public list browsing, token acquisition, queue management, and conversation replay.

Luckrig uses a three-layer protection mechanism:

1. Local Regex Filtering: The node proxy layer has a built-in filter to detect and block inputs in real time.
2. External Audit Hook: Configure an external audit service compatible with OpenAI Moderation via environment variables. Inputs are blocked before being sent; outputs are in default recording mode (recorded after streaming), which can be switched to blocking mode.
3. Reporting and Removal: Provides an abuse reporting endpoint. Operators manually ban after review to avoid automated misjudgments. In addition, illegal content such as child sexual exploitation is prohibited. A legal compliance checklist is provided, and it is recommended to consult legal professionals before deployment.

Differences between Luckrig and well-known projects:

• vs. AI Horde: AI Horde uses a kudos mechanism to abstract worker nodes and hide hardware details; Luckrig displays hardware metadata and tuning notes, allows specifying devices, sorts based on scarcity, and emphasizes hardware uniqueness and contribution value.
• vs. Petals: Petals focuses on distributed inference for large-scale models (model sharding collaboration); Luckrig adopts a single-node single-inference architecture where each node completes tasks independently, with a different technical approach.

As of now, Luckrig has completed the POC v1 version, implemented in pure Node.js with functions including public list, health monitoring, SQLite/JSONL persistence, token quota management, prompt filtering, node proxy, subtext encryption, queue visualization, multi-axis contribution scoring, Showcase generation, and conversation replay saving. Future plans: field validation, v6 feature development (multimodal such as image generation and voice processing), and operational enhancement; currently only text generation is supported, and other modalities need to wait for the confirmation of the pilot UX design.

Participation Methods: Those contributing computing power can join the network by completing node registration and proxy configuration via the CLI tool; users can browse the node list, obtain tokens, and wait for inference results through the Web interface, with the system supporting sorting by scarcity score. Conclusion: Luckrig is a new idea for decentralized AI infrastructure sharing, based on community trust and contribution spirit. It is not a competitor to traditional cloud computing or a resource exchange market, but an experimental platform. It has reference value in the path of computing power democratization and AI inclusiveness. Although it is in the POC stage, its concepts and implementation provide valuable experience for the distributed AI ecosystem.