Ensemble Enhancement of Weak Reasoning Models: How Multi-Agent Systems Achieve Performance Leaps

This article explores the core question: Can multiple weak reasoning models match the performance of a strong model through ensemble? The study uses a validator-supported committee search mechanism; 8 proposals from GPT-5.4 nano, after orchestration by a critique-comparator, achieved a 76.4% resolution rate on SWE-bench, matching the standalone performance of top-tier models. Key insight: Ensemble effectiveness does not depend solely on the number of agents, but rather on effectively identifying the correct solutions among the proposals from weak models.

In the field of large language models, there has long been an intuition: Can combining multiple weak models achieve the performance of a single strong model? This study focuses on reasoning models and explores the feasibility of validator-supported committee search as an in-reasoning enhancement mechanism. It challenges traditional perceptions: The mechanism is not simply "more agents are more helpful"; instead, it needs to identify correct solutions via critics and comparators when there is no access to a hidden validator.

The study establishes a formal framework, decomposed into four dimensions: proposal coverage, local identifiability, progressiveness, and diversity. Coverage can be amplified via repeated sampling, but coverage alone is insufficient to create effective critics/comparators; reliable performance amplification requires additional local reliability signals (e.g., execution results, proof checks, tests, etc.).

The study provides rank-based theoretical bounds, showing how local selection errors can combine into reliable trajectories. It also characterizes the upper limit of the proposal side: The convergence point of oracle best-of-k is limited to the set of task slices to which the proposal system assigns a non-zero useful probability—meaning the performance improvement of a perfect selection mechanism has a ceiling, which depends on the inherent quality of the proposal pool.

Experimental results on the SWE-bench Verified dataset: A single GPT-5.4 nano solved 67.0% of tasks; 8 proposals from the same model, after orchestration by a critique-comparator, achieved a resolution rate of 76.4%—matching the standalone performance of Gemini 3 Pro and Claude Opus4.5 Thinking, and approaching the theoretical upper limit of 79.0% for oracle best-of-8.

Core finding: Weak models can already generate a large number of correct solutions; the key lies in identification and selection. The critique-comparator mechanism successfully demonstrates that high-quality results can be extracted from weak model outputs through carefully designed validation and comparison processes. This is of great significance for reducing deployment costs—without relying on expensive top-tier models, optimizing the selection mechanism can unlock the potential of weak models.

The study analyzes remaining failure cases, which mainly stem from insufficient proposal coverage (shared blind spots). A stronger selection mechanism alone cannot compensate for the fundamental flaws of the proposal pool; future work needs to simultaneously improve proposal quality and optimize the selection mechanism.

This work has far-reaching implications for AI system design and deployment: Through an intelligent ensemble architecture, it significantly improves the practical performance of weak models, providing new ideas for building more cost-effective reasoning systems. Enterprises can reduce computing costs while achieving performance close to top-tier models, promoting the implementation of AI technology in a wider range of scenarios.