Code as Agent Harness: Building Executable, Verifiable, Stateful AI Agent Systems

This review proposes a new perspective of 'Code as Agent Harness', pointing out that code in agent systems has shifted from being a generation target to an operational foundation (infrastructure that connects, controls, and coordinates components). This paradigm leverages the executability, verifiability, and structural nature of code to provide a unified framework for building reliable and transparent agent systems, with a core three-layer architecture including interface, mechanism, and extension layers, pointing the way for the architecture of next-generation AI systems.

In traditional code generation tasks, LLMs generate code as the final product; however, in agent systems, code becomes the operational infrastructure—used for planning actions, interacting with the environment, maintaining state, and verifying results. The significance of this shift lies in the executability, verifiability, and structural nature of code, which are more reliable and transparent compared to natural language reasoning.

Interface Layer

Connects agents with reasoning, action, and environment modeling, acting as a bridge (e.g., automated scripts controlling applications, data analysis code processing experimental data). The challenge lies in flexible and secure interface design.

Mechanism Layer

Covers planning (breaking down tasks into code steps), memory (persisting state), tool usage (calling APIs), long-term execution, and feedback-driven control to ensure the system is reliable and adaptive.

Extension Layer

Extends to multi-agent scenarios, enabling coordination, review, and verification through shared code artifacts (e.g., jointly maintaining code repositories, code review mechanisms).

The paper covers multiple application domains:

• Programming assistants: Understanding code repositories, generating fix patches, executing tests;
• GUI/OS automation: Generating scripts to control desktop applications, managing file systems;
• Embodied agents: Core of the perception-action loop in robots/virtual environments;
• Scientific discovery: Generating data analysis and simulation code to assist research;
• Personalized recommendations: User behavior modeling and strategy generation;
• DevOps: CI/CD process automation;
• Enterprise workflows: Coordinating business steps and system interactions.

1. Evaluation beyond task success rate (focusing on intermediate quality such as code readability and efficiency);
2. Verification under incomplete feedback;
3. Harness improvement without regression;
4. State consistency in multi-agent systems;
5. Human supervision in safe operations;
6. Extension to multi-modal environments.

This review provides a new perspective on the role of code in agent systems, leveraging the characteristics of code to build more reliable and transparent AI systems. With the improvement of LLM code capabilities and the deepening of agent applications, 'Code as Harness' is expected to become the foundational methodology for next-generation AI agent systems, helping developers design robust and efficient architectures.