# GenCELLAgent: A Training-Free Multi-Agent System for Cellular Image Segmentation

> A multi-agent collaboration framework based on large language models, enabling zero-shot cellular image segmentation. It supports multiple microscopic imaging modalities and organelle types without requiring retraining for specific datasets.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-30T18:04:27.000Z
- 最近活动: 2026-05-30T18:20:10.434Z
- 热度: 155.7
- 关键词: 细胞图像分割, 大语言模型, 多智能体系统, 零样本学习, 显微成像, 医学图像处理
- 页面链接: https://www.zingnex.cn/en/forum/thread/gencellagent
- Canonical: https://www.zingnex.cn/forum/thread/gencellagent
- Markdown 来源: floors_fallback

---

## Introduction / Main Floor: GenCELLAgent: A Training-Free Multi-Agent System for Cellular Image Segmentation

A multi-agent collaboration framework based on large language models, enabling zero-shot cellular image segmentation. It supports multiple microscopic imaging modalities and organelle types without requiring retraining for specific datasets.

## Original Authors and Source

- **Original Author/Maintainer:** yuxi120407
- **Source Platform:** GitHub
- **Original Title:** GenCELLAgent: Generalizable, Training-Free Cellular Image Segmentation via Collaborative and Self-Evolving Large Language Model Agents
- **Original Link:** https://github.com/yuxi120407/GenCELLAgent
- **Release Date:** October 2025 (Last Updated: May 30, 2026)

## Background and Motivation

In cell biology research, researchers often need to process image data from different experimental conditions, imaging devices, and staining methods. While traditional deep learning segmentation models (e.g., Cellpose, SAM) perform well on specific datasets, they often require re-collecting annotated data and fine-tuning when faced with new imaging modalities or cell types.

The core insight of GenCELLAgent is: **Large language models have strong reasoning and planning capabilities, which can coordinate multiple specialized segmentation tools and dynamically select optimal strategies based on image features**. This approach eliminates the dependency on training data and achieves true zero-shot generalization.

## System Architecture: Three-Agent Collaboration Framework

GenCELLAgent adopts a modular multi-agent architecture, consisting of three core components that work collaboratively:

## 1. Planning Agent (Planner)

The Planning Agent is based on the Google Gemini large language model and is responsible for analyzing the features of input images and user natural language instructions. It formulates segmentation strategies and selects appropriate workflows based on image styles (phase contrast, fluorescence, confocal, electron microscopy, tissue sections, etc.) and target tasks.

## 2. Execution Agent (Executor)

The Execution Agent is responsible for calling specific segmentation tools. The system integrates multiple specialized models:
- **Cellpose**: A segmentation model optimized for cell morphology
- **micro-SAM / CellSAM**: SAM-adapted versions for microscopic images
- **SAM3**: The latest "segment anything" model
- **SegGPT**: Supports reference image-guided one-shot segmentation

The Execution Agent automatically selects and runs the most suitable tool for the current task based on the Planning Agent's decisions.

## 3. Evaluation Agent (Evaluator)

The Evaluation Agent uses a quality scoring mechanism based on Vision-Language Models (VLM) to iteratively optimize segmentation results. It checks metrics such as the accuracy of segmentation boundaries and cell integrity, and passes feedback to the Planning Agent for strategy adjustment.

## Tool Orchestrated Segmentation

Unlike the fixed behavior of a single model, GenCELLAgent integrates multiple domain-specific and general-purpose models into a collaborative framework. The system dynamically routes tasks based on image features, achieving complementary advantages among different models.