ThreadWeaver: Enabling Parallel Reasoning for Large Language Models Like Weaving

Meta and UC Berkeley jointly launched the ThreadWeaver framework, which reduces inference latency by 1.53x without sacrificing accuracy through adaptive parallel reasoning technology, opening a new path for optimizing large model inference efficiency. The core of the framework is to decompose serial reasoning into parallel 'threads', while exploring different solution paths for the problem and merging the results.

As large language models' capabilities improve, inference computational overhead increases. Mainstream autoregressive generation (decoding tokens one by one in sequence) leads to latency proportional to output length, with complex tasks requiring tens of seconds of waiting. Traditional optimizations (quantization, pruning, speculative decoding) have not addressed the structural bottleneck of serial decoding, making breaking through this a focus.

The core idea of ThreadWeaver is to decompose serial reasoning into parallel 'threads'. The technical architecture includes three components: 1. Parallel trajectory format (using tags like to organize reasoning structure, ensuring thread independence); 2. Five-stage inference state machine (compatible with existing optimization techniques such as prefix caching); 3. Trie-based training and P-GRPO reinforcement learning (avoiding information leakage, stably optimizing accuracy and speed).

In six mathematical reasoning benchmark tests, ThreadWeaver (taking Qwen3-8B as an example) has accuracy close to the serial baseline (e.g., AIME24 reaches 79.9% vs baseline's 78.3%). Latency is reduced by an average of 1.53x, with a maximum speedup of 1.92x on OlympiadBench. Real-scenario test (4 GPUs, 50 MATH500 questions): serial time takes 162.34 seconds, parallel only 142.21 seconds, speedup of 1.14x.

High-quality parallel data is key. Two-stage generation: 1. Rewrite existing serial reasoning chains using strong models to eliminate thread dependencies; 2. Self-training expansion, with format and answer filtering. The training set expands from 1k to 17k, format correctness rate increases from 56.4% to 77.0%, and accuracy reaches 79.9% after RL training.

Limitations: May produce redundant computations (e.g., repeated verification); mainly targeted at mathematical reasoning, need to migrate to scenarios like code generation and long text creation. Future directions: Combine with chain-of-thought and tool usage to enhance practical value.

ThreadWeaver represents an important direction for LLM inference from sequential generation to structured parallel exploration. Its open-source implementation provides a technical foundation for the community, and we look forward to more innovations emerging.