Active-VLM: Enhancing the Reasoning Capability of Vision-Language Models via Active Learning

Active-VLM introduces the concept of sequential experimental design, allowing vision-language models (VLMs) to actively select the most valuable data for learning. It aims to address issues like data redundancy and high annotation costs in traditional VLM training, significantly improving reasoning efficiency and accuracy. This article will cover its background, methods, experimental results, and other aspects.

Vision-language models (such as GPT-4V, Claude3) exhibit strong multimodal understanding capabilities, but traditional training requires massive amounts of image-text paired data with high annotation costs. Moreover, large volumes of data may be redundant, simplistic, or even lead to model biases or incorrect associations. Core question: Can we let models actively select the most valuable samples for learning? This is the starting point of Active-VLM.

Active learning allows models to actively query the most informative samples. Reasons why VLMs particularly need active learning: 1. Complex multimodal alignment (semantic gap between pixel features and language concepts); 2. Diverse reasoning paths (need to identify effective strategies); 3. Long-tailed data distribution (edge cases are scarce but critical).

Active-VLM transforms active learning into a sequential experimental design problem, consisting of three key components:

1. Uncertainty-guided sample selection: Integrates three types of uncertainty—visual (clarity of image understanding), language (semantic ambiguity of questions), and reasoning (path uncertainty)—to calculate information value scores.
2. Diversity-aware batch selection: Uses core set methods to ensure sample coverage, balancing information value and similarity.
3. Adaptive query strategy: Dynamically adjusts the sample selection strategy based on training stages (early-stage basic alignment, mid-stage boundary expansion, late-stage reasoning refinement).

Active-VLM not only selects data but also optimizes learning methods:

1. Chain-of-thought reinforcement: Explicitly models reasoning steps, learning both final answers and intermediate processes to enhance structured reasoning capabilities.
2. Contrastive reasoning learning: Generates multiple candidate reasoning paths, distinguishes between correct and incorrect paths, and understands reliable reasoning patterns.
3. Multimodal attention calibration: Encourages models to focus on image regions relevant to the answer, alleviating the "hallucination" problem.

Active-VLM performs excellently in benchmark tests:

• Data efficiency: Maintains the same performance with only 30%-50% of the annotated data required by traditional methods.
• Reasoning accuracy: Improves by 5-10 percentage points in complex tasks (e.g., visual question answering), with more significant gains on out-of-distribution test sets.
• Robustness: More robust to adversarial samples and noisy inputs, avoiding learning "shortcut" features.

Application Value: Reduces annotation costs, improves model quality (friendly to resource-constrained scenarios), and supports continuous learning (selects valuable samples for incremental training). Limitations: Uncertainty estimation has noise, and sample selection increases computational overhead. Future Directions: Expand to multimodality (video, audio), combine reinforcement learning to optimize exploration-exploitation trade-offs, and develop efficient approximation algorithms to accelerate sample selection.