Synapse: Implementing Distributed Large Model Inference via Browsers, Turning Every Device into a Computing Node

Synapse is a revolutionary distributed inference engine. It uses WebGPU technology to split large language models across multiple browsers and devices for execution, eliminating the need for cloud GPUs or API keys. This allows ordinary devices like phones, tablets, and laptops to collaboratively complete AI inference tasks, with the goal of turning the internet itself into a supercomputer.

In the era of large AI models, computing power is scarce. Calling cloud APIs or renting GPUs requires continuous financial investment. However, the GPUs of billions of devices worldwide (smartphones, tablets, etc.) are idle most of the time. Based on this insight, Synapse proposes a distributed inference paradigm where browsers act as computing nodes.

Synapse's technical architecture consists of five components: Model Splitting (splitting HuggingFace models into N shards), Local Loading (browsers download and cache shards), Parallel Computing (implementing core operators with custom WGSL shaders), Efficient Routing (SYN1 binary protocol supporting int8 quantization), and Autoregressive Generation (efficient generation via KV caching). Optimizations focus on reducing network transmission; binary protocol and KV caching optimizations have already improved performance by 15x, and future plans include attention head pruning and WebRTC P2P.

On April 13, 2026, the project demonstrated the collaborative running of the GPT-2 model on a Pixel 10 Pro XL and an iPhone 16 Pro. Coordinated via a low-cost GCP virtual machine, it successfully generated 15 tokens at a speed of 1.3 tokens per second. Deployment steps are simple: clone the repository, install dependencies, split the model, start the coordinator, and open a browser tab to become a node for collaborative inference.

Synapse's application scenarios include: Education (classroom Chromebooks collaborating to provide AI-assisted learning), Home (family phones forming a cluster to serve smart homes), Edge Computing (cross-internet browser grids providing low-cost AI services to remote areas), similar to the SETI@home project in the field of AI inference.

Synapse proves that ordinary devices can collaboratively complete complex AI tasks, lowering the barrier to AI usage while ensuring data privacy (inference is done locally). As WebGPU becomes more popularized in the future, this paradigm may become part of AI infrastructure. Developers are encouraged to participate in the open-source project and explore more possibilities of distributed AI.