Self-Improving Reasoning Agent: Achieving Self-Evolution of Reasoning Capabilities via Dual-Model Architecture

This article introduces the open-source project Self-Improving-Reasoning-Agent, which achieves self-detection and correction in AI reasoning processes through a two-stage collaborative architecture of generative and evaluative models, significantly enhancing reasoning reliability in complex tasks. Developed by ahmadbuilds, the project uses a modern tech stack and supports multiple deployment methods.

Large Language Models (LLMs) excel in text generation, but they often have mathematical errors or logical loopholes (hallucinations) in complex reasoning tasks, limiting their application in high-precision scenarios. Developer ahmadbuilds launched this project to build a reasoning evaluation pipeline; its core innovation is a two-stage architecture: a base LLM generates reasoning answers, and a specially trained evaluative model detects and classifies errors, enabling iterative self-correction.

The project uses a separate frontend-backend architecture with a tech stack including Python, TypeScript, TensorFlow, FastAPI, Next.js, etc. Core modules: backend (data processing, model training, FastAPI interfaces), frontend (Next.js 16 frontend with Tailwind CSS + Framer Motion), and Dockerfile supporting deployment on Hugging Face Spaces. Backend components include Data, Notebooks, Reports, Trained_Weights, main.py, etc.

The generative model is responsible for receiving questions and generating structured reasoning chains. It supports fine-tuning of TinyLlama and Phi-2, and currently integrates the Groq LLaMA API, outputting a standardized format (question, reasoning process, answer). The evaluative model is based on the DeBERTa-v3 architecture, fine-tuned via Keras-Hub, lightweight and efficient, identifying three types of errors: mathematical calculation errors, logical reasoning errors, and missing reasoning steps.

The dataset uses the GSM8K primary school math reasoning dataset plus a synthetic error dataset. Preprocessing steps: cleaning, error injection, and format standardization into quadruples. Training uses labeled samples (whether reasoning is correct and error type) with a cross-validation strategy. Evaluation metrics: accuracy, precision, recall, F1 score. Training reports show that the DeBERTa model converges stably, and confusion matrices and F1 curves verify its classification performance.

The FastAPI backend orchestrates the reasoning pipeline, handles errors, provides metric data, and loads tokenizers and models to achieve sub-second responses. The Next.js 16 frontend uses Tailwind CSS for responsive layouts, Framer Motion for animations, and ReasoningBlock components to display reasoning. Deployment supports local development, Docker containerization, and Hugging Face Spaces (note: model weights are managed with Git LFS).

Applicable scenarios include education (math problem-solving verification), scientific research (paper logic checking), code review (logical loophole detection), and intelligent customer service (complex problem answering). Core contribution: Proving the feasibility of lightweight evaluative models supervising large generative models, providing a new path for reliable AI systems.

The project addresses the reasoning reliability issue of LLMs through a dual-model architecture. It has clear code, complete documentation, and easy deployment, providing a reproducible and extensible reasoning evaluation framework. Future directions: Expand to more reasoning domains (code, science), co-train generative and evaluative models, introduce reinforcement learning optimization strategies, and support multi-modal reasoning evaluation.