E3-TIR: A New Paradigm for Agent Training in Tool-Integrated Reasoning

This article introduces E3-TIR (Enhanced Experience Exploitation for Tool-Integrated Reasoning), a new paradigm for agent training. Its core lies in integrating three types of experience: expert prefixes, expert guidance, and self-exploration, aiming to solve the problems of low exploration efficiency and high data costs in tool-integrated reasoning training. Experiments show that this paradigm achieves a 6x performance improvement, a 90% reduction in data requirements, and a 1.46x increase in ROI. The following floors will elaborate on the background, methods, experimental results, and other content.

Tool-Integrated Reasoning (TIR) is a core capability of AI agents, enabling them to call external tools to assist in reasoning and solve complex tasks. However, existing training paradigms face challenges: Zero-RL methods have low exploration efficiency and are prone to local optima; SFT-then-RL methods have high data costs and easily fall into a performance plateau due to low-entropy collapse.

E3-TIR dynamically integrates experience through three stages: 1. Expert Prefix: Learn key decision points in expert trajectories to quickly establish basic cognition of tool usage; 2. Expert Guidance: Conduct branch exploration centered on expert anchors to balance direction and diversity; 3. Self-Exploration: Encourage breaking out of the expert framework to expand knowledge boundaries, complementing expert guidance.

E3-TIR introduces a hybrid strategy optimization mechanism. It alleviates distribution shift issues by dynamically adjusting the weights of experiences from different sources, and uses a hierarchical credit assignment mechanism to resolve optimization conflicts of shared prefixes, ensuring the model learns stably from diverse experiences.

Experimental results show: E3-TIR achieves a 6x performance improvement compared to traditional paradigms; reduces synthetic data requirements by more than 90%; and increases the comprehensive ROI index by 1.46x, demonstrating its significant advantages in performance, data efficiency, and resource investment returns.

E3-TIR breaks through existing training bottlenecks and provides a new paradigm for agent training. Its idea of combining experts and exploration can be applied to human-machine collaboration scenarios such as multi-tool collaboration and long-term task planning, providing a direction for efficiently training high-performance agents under resource constraints.