AIR: Adaptive Interleaved Reasoning and Code Collaboration Framework for Multimodal Large Language Models

The AIR (Adaptive Interleaved Reasoning) framework deeply integrates code execution with multimodal understanding through an adaptive interleaved reasoning mechanism, aiming to solve the information integration problem of Multimodal Large Language Models (MLLMs) in complex reasoning tasks and significantly enhance their solving ability. Its core lies in breaking the linear reasoning process, dynamically switching modalities and using code as an intermediate representation, combining adaptive decision-making and code collaboration to provide practical experience for neuro-symbolic fusion.

With the development of MLLMs, processing multimodal inputs such as text and images has become a trend, but integrating multimodal information and generating reliable reasoning chains in complex reasoning remains a core problem. The traditional linear process (first visual understanding then language reasoning) performs poorly in multi-step collaborative tasks, so the AIR framework proposes an adaptive interleaved reasoning paradigm.

AIR breaks the limitations of traditional linear reasoning and introduces the concept of 'interleaved reasoning'—dynamically switching modalities according to task requirements and expressing intermediate results in code form for execution. This design can reduce semantic drift, expand capabilities with external tool verification, and flexibly adjust the depth and breadth of reasoning.

The adaptive feature is key to AIR. Through a lightweight decision module, it evaluates the current state (confidence, coherence, remaining complexity, modal complementarity) and decides the next step (deepen current modality, switch modality, generate code for execution), balancing reasoning quality and computational cost.

Code is an important part of the AIR reasoning process. It converts intermediate results into executable code (such as Python) to achieve precise calculation, data processing, and logical verification. Executable code feedback corrects the reasoning direction, modularly decomposes tasks, and error information provides learning signals to improve accuracy.

AIR performs excellently in multimodal reasoning benchmark tests, especially showing obvious advantages in tasks combining visual understanding and mathematical reasoning (geometric problem solving, chart analysis, scientific experiment explanation). It provides new ideas for neuro-symbolic fusion for researchers, open-source references for developers, and reliable help for end-users in more complex tasks.

AIR marks a new stage in MLLM reasoning research. Adaptive interleaved reasoning can be extended to tool usage such as database queries and API calls. Future directions include improving the intelligence of the decision module, integrating more modalities (video, 3D, sensor data), and applying it to real-time interaction scenarios (robot control, autonomous driving). AIR opens a new path for the development of MLLMs and provides practical experience for neuro-symbolic fusion.