PaCT: A Parallel Continuous Thinking Reasoning Architecture for Large Language Models

PaCT proposes a novel parallel continuous thinking mechanism. By breaking the serial constraints of traditional chain-of-thought reasoning, it enables large language models to explore multiple reasoning paths simultaneously, significantly improving the efficiency and accuracy of complex reasoning tasks. This work is a submission to ICML 2026, developed and open-sourced by jithamanyu001 on GitHub (link: https://github.com/jithamanyu001/pact), with a release date of 2026-05-26.

Large language models face challenges in complex reasoning scenarios: The traditional Chain-of-Thought (CoT) prompting technique is inherently serial, requiring step-by-step generation of intermediate steps and unable to explore different paths in parallel. Its issues include: long reasoning time; a single error leading to subsequent deviations; and a single path limiting the use of multiple problem-solving strategies.

The PaCT (Parallel Continuous Thinking) architecture breaks the linear constraints of traditional chain-of-thought, allowing the model to maintain and develop multiple parallel thinking threads simultaneously. Drawing on the cognitive strategy of humans considering multiple possibilities when facing complex problems, it formalizes this into a computable architecture, enabling the model to evaluate multiple reasoning hypotheses in parallel during a single forward pass or limited iterations.

The technical implementation of PaCT includes three key components: 1. Parallel thinking space construction: The model encodes multiple candidate reasoning states simultaneously at the implicit representation level; 2. Continuous thinking state management: Introduces continuous state representation, where the reasoning process evolves smoothly in a continuous semantic space, facilitating gradient propagation and path fusion/branching; 3. Path selection and pruning strategy: Dynamically retains promising branches and prunes deviant paths through an intelligent evaluation mechanism, balancing scalability and efficiency.

PaCT has application potential in multiple domains: For mathematical problem solving, it can try multiple strategies in parallel; for code generation, it can explore different algorithms and data structures; for scientific reasoning, it can evaluate multiple hypotheses simultaneously. In high-reliability scenarios, the multi-path feature provides uncertainty quantification: Convergent paths indicate high confidence, while divergent paths trigger further analysis or human intervention.

Compared with Tree of Thoughts (ToT): ToT expands multiple paths with discrete nodes, with limited information sharing between paths; PaCT's continuous state representation enables finer-grained interaction and knowledge sharing. Compared with self-consistency decoding: The latter samples independent answers for voting, while PaCT allows multiple threads to influence each other and evolve together during reasoning, improving performance stability.

The open-sourcing of PaCT facilitates the reproduction and verification of experimental results, laying the foundation for community improvements and applications. Future research directions include: deep adaptation to specific model architectures; exploring performance on large-scale models; developing task-specific variants; and studying the interpretability of parallel reasoning (explicit representation of multiple paths helps understand the internal mechanisms of the model).

PaCT is an important attempt in the evolution of reasoning enhancement technology from 'serial single path' to 'parallel multi-path'. While maintaining the general capabilities of large language models, it significantly improves complex reasoning performance. With the release of the open-source code, we look forward to its application and verification in more practical scenarios.