Miru: A Visual Tracking Tool for Multimodal Reasoning Processes

Miru is an open-source multimodal reasoning tracking tool based on FastAPI, designed to solve the "black box" dilemma of multimodal models like GPT-4V and Claude 3. It can generate step-by-step reasoning trajectories, label the image regions or text paragraphs relied on by each reasoning step of the model, and provide an interactive attention visualization feature to enhance the interpretability and credibility of AI systems.

With the popularity of vision-language large models like GPT-4V and Claude 3, multimodal AI can now understand and analyze image content, but these models often lack transparency when giving answers—users cannot know which region of the image or which paragraph of the document the model based its judgment on. This "black box" characteristic is particularly worrying in high-risk scenarios such as medical diagnosis and legal analysis.

1. Step-by-Step Reasoning Tracking

Generate "reasoning trajectories" that record the model's thinking process at each reasoning step, allowing users to understand the path from original input to conclusion by the AI.

2. Interactive Attention Visualization

Present the model's attention mechanism with heatmaps or highlighted areas, clearly showing the image regions or document paragraphs the model focuses on when answering questions.

3. FastAPI Backend Architecture

Adopts the FastAPI framework, which has advantages of high performance, asynchronous processing, and automatic API documentation generation, making it easy to deploy and integrate into existing multimodal application pipelines.

Miru's technical implementation involves:

• Attention mechanism extraction: Intercept the intermediate layer output of multimodal models to capture attention weight distribution
• Region-reasoning association: Establish mapping between image regions/text fragments and specific reasoning steps
• Trajectory structuring: Organize scattered attention information into human-readable reasoning chains
• Visualization rendering: Convert abstract attention data into an intuitive graphical interface

Medical Image Analysis

Assist doctors in verifying the reliability of AI diagnoses and understanding which features of the lesion the model based its judgment on.

Document Review and Compliance

Show the specific location of the clauses cited by the model, improving the auditability of legal/contract review results.

Education and Research

Help researchers and students understand the internal mechanisms of multimodal models, promoting learning in the XAI field.

Model Debugging and Optimization

Locate the root cause of erroneous reasoning and improve visual/text features that the model easily confuses.

Miru represents an important exploration of XAI in the multimodal field. As AI is deployed in critical scenarios, "explainability" is changing from a bonus to a necessity. It provides a practical solution to the black box problem of multimodal AI, enhances user trust, and provides diagnostic information for model improvement—it is an open-source project worth paying attention to.