Simple-VTIR-Agent: A Practice of Lightweight Visual Tool-Integrated Reasoning Agent

Simple-VTIR-Agent is a lightweight visual tool-integrated reasoning agent based on Kimi K2.6. It implements multi-round tool calls and visual understanding through a local IPython environment, demonstrating the application paradigm of vibe-coding (intuitive coding) in agent development. As a learning prototype, the project simplifies the SWE-Vision framework, focusing on the readability and debuggability of the core reasoning loop to help developers quickly understand the working principles of VLM Agents.

With the improvement of multi-modal large language model (VLM) capabilities, the combination of visual understanding and tool calling has become an important direction in AI application development. This project was quickly built via vibe-coding, inspired by the open-source framework SWE-Vision but greatly simplified—removing complex Docker containerization and web interfaces, focusing on the readability and debuggability of the core reasoning loop. It is suitable for developers to quickly understand the principles of VLM Agents and conduct experimental development.

Simple-VTIR-Agent follows the classic visual tool-integrated reasoning paradigm, with the workflow including the following key links:

1. User input processing: Users upload images via the command line and attach task instructions. The system copies the images to the working directory and encodes them into the base64 image_url format compatible with Kimi K2.6.

2. Multi-round reasoning loop: The core of the agent is a continuous dialogue loop. In each round, Kimi K2.6 analyzes the images and dialogue history to decide whether to call the code execution tool.

3. Local code execution environment: A local IPython environment is used as the backend. If it is unavailable, it falls back to the exec function, sacrificing isolation for efficiency and debugging convenience.

4. State persistence and tracking: An independent working directory is created for each run, storing images, dialogue records, and intermediate files to facilitate retracing the reasoning process.

The core tool of the project is execute_python, which accepts Python code strings for execution and returns outputs, errors, and generated images. It interacts with Kimi K2.6 through an OpenAI-compatible API, with requests including system prompts, user messages (images + text), and historical records. Note that local execution has no sandbox isolation; it is only suitable for trusted experimental environments and not recommended for production deployment.

The project applies to multiple scenarios:

• Image analysis and measurement: Upload charts/design drafts/scientific images and generate OpenCV/PIL code to complete pixel-level measurement, data extraction, or pattern recognition.

• Multi-image comparison analysis: Support multi-image upload, write code for pixel comparison, difference detection, or change tracking.

• Mathematical logic calculation: Use Python's precise numerical calculation capabilities to solve complex mathematical problems.

• Educational learning: The concise code structure helps developers master the core concepts of VLM Agents.

The project provides a static frontend viewer that supports: filtering messages by role, expanding/collapsing the reasoning process, displaying code blocks with syntax highlighting, rendering base64 images, full-text search, and image scaling. Developers can start the viewer via an HTTP server to review the complete interaction process of any run.

As a learning prototype, it has the following limitations:

1. The local execution environment lacks isolation and has security risks. It is recommended to refer to SWE-Vision for using Docker containerization solutions.

2. Only single-round dialogue batch processing mode is supported; there is no continuous interactive chat interface, so the experience for complex tasks is not smooth enough.

3. The tool interface is single, only supporting Python code execution. It is necessary to expand tool types such as web search, database query, and API calls.

Simple-VTIR-Agent demonstrates the potential of vibe-coding in AI tool development. By focusing on core functions, keeping code concise, and being transparent about trade-offs, a usable prototype can be quickly built. The value of the project lies in providing an entry point for the community to understand VLM Agents, making it an excellent learning resource for getting started with multi-modal AI development. In the future, lightweight agent frameworks will promote the inclusive implementation of AI in more vertical fields.