Impetus: Injecting Energy to Optimize the Inference Layer for Open-Source Large Models

The Impetus project explores applying Energy-Based Models (EBM) to enhance the inference of open-source large language models. It improves mathematical and logical reasoning abilities through two phases—candidate reordering and latent space optimization—without retraining the base model. The project aims to achieve measurable performance improvements on benchmarks like GSM8K and ARC, providing the open-source community with a new path to efficiently utilize existing model capabilities.

Current mainstream large language models use autoregressive generation. While word-by-word prediction is efficient, it has limitations in complex reasoning tasks such as hallucinations, logical breaks, and local optima—especially noticeable in mathematical and logical judgments. Impetus proposes a core hypothesis: introducing an energy optimization layer after generation, selecting the optimal reasoning path through evaluation, can significantly improve reasoning quality without modifying the base model itself, serving only as a post-processing enhancement layer.

An Energy-Based Model is a neural network that maps inputs to a scalar "energy value". A lower energy value indicates a more reasonable sample. In Impetus, the system calculates an energy score for each candidate reasoning path and selects the answer with the lowest score as the output. Unlike traditional autoregressive generation, EBM adopts a "generate first, select later" strategy, allowing global evaluation of multiple candidate responses and avoiding irreversible local decisions.

Impetus adopts a progressive development strategy, divided into two phases:

Phase 1 (V1: Candidate Reordering)

After the base model generates multiple candidate responses, three energy scoring methods are used to reorder and select the optimal one:

• Self-consistency method: The model evaluates the consistency of its own generated answers
• Embedding consistency method: Calculate the semantic similarity between the problem, reasoning process, and answer
• Lightweight neural network EBM: Train a small scoring network to evaluate the quality of reasoning paths

Phase 2 (V2: Latent Space Optimization)

If the effect of V1 is positive, we will explore modifying the hidden state before decoding, optimize the model representation through iterative energy minimization, and pursue more fundamental improvements.

The project uses scientific and rigorous experimental methods:

• Benchmark Tests: Prioritize GSM8K (mathematical word problems), ARC (scientific reasoning), and BBH (large model benchmark). After verifying the effect, expand to hallucination detection and factuality evaluation
• Control Experiments: Compare with baseline models, report benchmark scores and latency metrics, and reject subjective evaluations
• Goal Setting: Minimum goal is a 3-5% improvement without significant latency increase; ideal goal is an 8-12% improvement.

The project uses lightweight open-source models to ensure reproducibility and low cost:

• Models: Alibaba Qwen 2.5-3B Instruct, Meta Llama 3B-8B variants, TinyLlama, SmolLM, and other small models
• Frameworks: PyTorch, Transformers, Accelerate, Datasets, Evaluate, BitsAndBytes, OpenCompass

The project does not pursue large model parameter scales; it focuses on verifying the effectiveness of the method.

Impetus represents a new research idea: instead of increasing model scale, it efficiently utilizes existing model capabilities. Energy-based models provide a new path for enhancing large model inference. If verified effective, it will provide the open-source community with a method to improve reasoning capabilities without retraining the base model—significant for researchers and developers with limited resources, opening up new possibilities for efficient use of large models. The core question of the project: "Can energy-based reasoning improve the performance of open-source large models in mathematical and logical tasks?" will be answered by experimental data.