Zing Forum

Reading

GenCELLAgent: A Training-Free Multi-Agent System for Cellular Image Segmentation

A multi-agent collaboration framework based on large language models, enabling zero-shot cellular image segmentation. It supports multiple microscopic imaging modalities and organelle types without requiring retraining for specific datasets.

细胞图像分割大语言模型多智能体系统零样本学习显微成像医学图像处理
Published 2026-05-31 02:04Recent activity 2026-05-31 02:20Estimated read 5 min
GenCELLAgent: A Training-Free Multi-Agent System for Cellular Image Segmentation
1

Section 01

Introduction / Main Floor: GenCELLAgent: A Training-Free Multi-Agent System for Cellular Image Segmentation

A multi-agent collaboration framework based on large language models, enabling zero-shot cellular image segmentation. It supports multiple microscopic imaging modalities and organelle types without requiring retraining for specific datasets.

2

Section 02

Original Authors and Source

  • Original Author/Maintainer: yuxi120407
  • Source Platform: GitHub
  • Original Title: GenCELLAgent: Generalizable, Training-Free Cellular Image Segmentation via Collaborative and Self-Evolving Large Language Model Agents
  • Original Link: https://github.com/yuxi120407/GenCELLAgent
  • Release Date: October 2025 (Last Updated: May 30, 2026)
3

Section 03

Background and Motivation

In cell biology research, researchers often need to process image data from different experimental conditions, imaging devices, and staining methods. While traditional deep learning segmentation models (e.g., Cellpose, SAM) perform well on specific datasets, they often require re-collecting annotated data and fine-tuning when faced with new imaging modalities or cell types.

The core insight of GenCELLAgent is: Large language models have strong reasoning and planning capabilities, which can coordinate multiple specialized segmentation tools and dynamically select optimal strategies based on image features. This approach eliminates the dependency on training data and achieves true zero-shot generalization.

4

Section 04

System Architecture: Three-Agent Collaboration Framework

GenCELLAgent adopts a modular multi-agent architecture, consisting of three core components that work collaboratively:

5

Section 05

1. Planning Agent (Planner)

The Planning Agent is based on the Google Gemini large language model and is responsible for analyzing the features of input images and user natural language instructions. It formulates segmentation strategies and selects appropriate workflows based on image styles (phase contrast, fluorescence, confocal, electron microscopy, tissue sections, etc.) and target tasks.

6

Section 06

2. Execution Agent (Executor)

The Execution Agent is responsible for calling specific segmentation tools. The system integrates multiple specialized models:

  • Cellpose: A segmentation model optimized for cell morphology
  • micro-SAM / CellSAM: SAM-adapted versions for microscopic images
  • SAM3: The latest "segment anything" model
  • SegGPT: Supports reference image-guided one-shot segmentation

The Execution Agent automatically selects and runs the most suitable tool for the current task based on the Planning Agent's decisions.

7

Section 07

3. Evaluation Agent (Evaluator)

The Evaluation Agent uses a quality scoring mechanism based on Vision-Language Models (VLM) to iteratively optimize segmentation results. It checks metrics such as the accuracy of segmentation boundaries and cell integrity, and passes feedback to the Planning Agent for strategy adjustment.

8

Section 08

Tool Orchestrated Segmentation

Unlike the fixed behavior of a single model, GenCELLAgent integrates multiple domain-specific and general-purpose models into a collaborative framework. The system dynamically routes tasks based on image features, achieving complementary advantages among different models.