GAIS: MCP-based Grounded Interaction Synthesis Framework Breaks Agent Data Bottleneck, Achieves Stronger Capabilities with Less Data

GAIS (Grounded Agent Interaction Synthesis Framework) addresses the agent data dilemma through a two-stage grounding mechanism (protocol-anchored environment + structure-guided planning) to build diverse environments from real MCP servers. Experiments show it outperforms the official instruction-tuned versions on BFCL, τ²-Bench, and ACEBench, achieving stronger capabilities with less data and providing a new direction for agent data synthesis.

General-purpose agents rely on high-quality interaction data, but manual annotation costs are extremely high (complex tasks require hours of annotation); LLM-synthesized data has issues like biased sampling (tending to common scenarios) and low fidelity (detached from reality), making it hard to support the development of complex agent capabilities.

The core of GAIS is anchoring real tool protocols: 1. Protocol-anchored environment construction: Connect to real MCP servers, integrate real tools, ensuring environment authenticity and diversity; 2. Structure-guided planning: Generate complex tasks via logical dependency graphs and adversarial strategies, introduce error recovery scenarios, and enhance task challenge.

In three benchmark tests—BFCL (function calling), τ²-Bench (tool usage), and ACEBench (comprehensive capability)—the base model + GAIS data matches or outperforms the official instruction-tuned versions; data efficiency is significant (stronger capabilities with less data), and performance grows continuously with increasing data volume, showing good scalability.

• Value of MCP protocol: Standardized interfaces reduce tool integration costs, connecting real services avoids environment trivialization, and benefits from an active community ecosystem; - Structure-guided planning: Ensures task complexity via logical dependency graphs, adversarial design enhances robustness, and supports long-range planning; - Data synthesis comparison: GAIS achieves the optimal balance in cost, authenticity, diversity, complexity, and scalability (compared to manual annotation and unconstrained LLM synthesis).

Applicable scenarios: Agent training data construction, tool usage evaluation, rapid integration of new tools, domain adaptation; Synergy with MCP ecosystem: MCP provides tool interfaces → GAIS generates data → promotes MCP adoption → ecosystem expansion feeds back to GAIS; Open-source contribution: Code repository https://github.com/Eric8932/GAIS, supporting community contributions and reproducibility.

Current limitations: Dependence on MCP protocol, challenges in modeling complex tools, limited multimodal support; Future directions: Expand multi-protocol support, improve online learning, introduce human feedback to optimize data, research cross-domain migration.

GAIS solves the LLM-synthesized data problem through real-world anchoring; experiments prove its effectiveness (outperforming tuning with less data), highlighting the value of the grounding methodology. With the development of the MCP ecosystem, GAIS will become a scalable and reproducible solution for agent data construction, inspiring the AI data synthesis field.