GEPA Evolutionary Multi-Agent Programming Framework: Enabling Fixed Models to Self-Iterate and Generate Stronger Code

This project implements the GEPA (Genetic Evolution of Prompting Architecture) evolutionary multi-agent programming framework. Using a fixed Claude Haiku model, it leverages a nested multi-agent architecture to self-iterate, automatically generating and validating stronger BattleSnake game AI code. This framework explores a new path for models to self-evolve prompt words and code architectures. Compared to traditional model upgrades or manual prompt engineering, it has the advantages of low cost and high automation.

In LLM application development, traditional code generation has two major limitations: 1) The model upgrade path is costly and has high latency; 2) Manual prompt engineering requires a lot of trial and error and experience.This project proposes a third path: enabling models to self-iterate and optimize code generation strategies through a genetic algorithm-style evolutionary mechanism, without the need for manual trial of prompt variants one by one.

The core idea of GEPA is to treat prompt words and code architectures as "genes", and evolve optimal solutions through operations such as selection, mutation, and crossover. Its key components include population, fitness function, selection, mutation, and crossover. The project adopts a nested multi-agent architecture: the meta-agent designs code generation strategies, sub-agents (strategy, implementation, testing, etc.) perform tasks, and finally, verification processes such as static checks, unit tests, and integration tests are carried out.

The project is built based on Claude Dynamic Workflows. The reasons for choosing the fixed Claude Haiku model include cost-effectiveness, low latency, capability boundary verification, and reproducibility. Dynamic workflows allow the meta-agent to dynamically create sub-agents. The pseudocode illustrates the evolution cycle: initialize population → evaluate each individual (design architecture → generate code → verify → calculate fitness) → evolve the next generation.

BattleSnake was chosen as the verification platform (simple rules but complex strategies, quantifiable evaluation, etc.). Evaluation metrics include win rate, average ranking, survival time, etc. Experimental results: The initial generation's win rate was about 15%, reaching 68% at the 100th generation; excellent architecture patterns include strategy-implementation separation, test-driven development, etc. Compared to manual prompt engineering, GEPA has a higher win rate and lower manual input.

Technical insights include: 1) Strict verification is key to evolutionary algorithms; 2) Population diversity avoids premature convergence;3) Hierarchical design allows each agent to focus on its abstract level;4) Fixed models can complete complex tasks through architecture optimization.

The GEPA framework can be extended to code tasks such as algorithm implementation, API encapsulation, and test generation, and can also be applied to non-code fields such as data analysis pipelines, content generation, and dialogue systems.

Current limitations: High computational cost, task dependence on fitness functions, verification bottlenecks, and convergence uncertainty. Future directions: Transfer learning, online evolution, integration of human feedback, multi-objective optimization, and expansion of the architecture search space.