ResearchFlow: An Index-First Workflow Framework for AI Agent-Assisted Research

ResearchFlow is an index-first workflow framework for AI agent-assisted research, aiming to address the context management challenges faced by agents in scenarios involving long conversations, heterogeneous files, and cross-project knowledge. Through an index-first reading strategy, auditable reasoning paths, and a layered knowledge architecture, it provides a systematic solution for agent-assisted research and project execution, effectively controlling token costs, mitigating memory drift, and enabling traceable reasoning processes.

With the widespread application of LLMs and AI agents in scientific research and software development, core challenges have become prominent: agents have limited context, while real research projects involve large amounts of heterogeneous files, long conversation histories, simulation experiment outputs, and cross-project knowledge accumulation, leading to surging token costs, memory drift, and difficulty in auditing reasoning processes. The traditional interaction mode of directly loading all documents has obvious bottlenecks in complex scenarios, requiring more efficient context management strategies.

The core concept of ResearchFlow is 'index-first', treating context as a scarce resource—agents first read compact index files and only open specific evidence files when needed for the task. Advantages include: controlling token costs (pre-screening reduces full document loading), mitigating memory drift (compact indexes reduce irrelevant interference), and auditable reasoning paths (tracking decision processes from indexes to files). The framework uses JSONL format to build high-density indexes, balancing machine parseability and human readability.

The framework constructs a four-layer knowledge management system: general method layer (cross-project methodologies), bridge layer (shared knowledge between projects), project layer (domain knowledge and status of specific projects), and external library layer (integration of third-party resources). The dual-mode adapter system is compatible with different projects: native mode (creates a .researchflow/ metadata directory in the project directory) and legacy mode (creates shadow adapters in the interface directory for zero-invasion integration), ensuring wide applicability.

The framework manages the agent's operation cycle through a compact state machine, including eight core phases: ingestion and classification (receiving intent and classifying tasks), planning (formulating execution plans based on indexes), execution (executing according to plans and dynamically adjusting), digestion and declaration (organizing results to form knowledge claims), review and memory (extracting project memories or framework evolution proposals after review). The structured workflow improves execution predictability, supporting knowledge precipitation and framework optimization.

The framework defines multiple reading strategies: index_only (read only indexes), on_demand (load on demand), adapter_first (prioritize adapters), registry_only (only skill registry), skip (skip resources). Privacy boundaries are layered: the public release boundary contains sanitized rules, manuals, etc.; the local private boundary retains sensitive information such as project facts and personal preferences, balancing knowledge sharing and privacy protection.

ResearchFlow provides an engineering solution for AI-assisted research: researchers solve context management problems in complex projects; developers obtain standardized agent collaboration protocols. The index-first idea can be applied to LLM application development, knowledge management systems, multi-agent collaboration architectures, etc., bringing performance and cost advantages. As model context windows grow, efficient use of context resources becomes key, and the framework provides a practice-verified solution.

ResearchFlow is an important exploration in the engineering of AI-assisted research workflows—it is not just a collection of tools but also a methodology for agent collaboration. Through index-first design, layered architecture, and state machines, it provides a feasible path for agent applications in complex scenarios. In the future, the enhancement of multimodal models and the maturity of the agent ecosystem will increase the framework's importance. Open-source release lays the foundation for community contributions and iterations, and it is expected to continue evolving in academia and industry.