Reconsidering Decoding Strategies in Visual Question Answering: Why Greedy Decoding May Be Better Than Random Sampling

Recent research indicates that in the visual question answering (VQA) task of multimodal large language models (MLLMs), the simple greedy decoding strategy may outperform complex random sampling methods. From the perspective of model calibration, the study reveals the essential differences in the sources of uncertainty between VQA tasks and text generation tasks: VQA is a closed-ended task, where uncertainty stems from cognitive levels (lack or ambiguity of visual evidence) rather than the need for diversity in text continuation.

In the development of large language models (LLMs), random sampling strategies (such as temperature sampling, Top-p sampling) are standard configurations, aiming to balance coherence and diversity. But are these methods from the pure text domain applicable to the VQA task of MLLMs? The study points out that there are essential differences between VQA and open-ended text generation: VQA is usually a closed-ended task with a concentrated answer distribution at the head, and uncertainty mainly comes from cognitive levels (difficulty in visual understanding) rather than the need for diversity.

The core contribution of the study is to establish a theoretical connection between model calibration and prediction accuracy. Model calibration refers to the consistency between output confidence and actual accuracy. In the VQA scenario, the answer space is limited and has clear standard answers, and uncertainty reflects the degree of understanding of the input. The team derived the sufficient conditions for the optimality of greedy decoding: when the model is well-calibrated and the task is closed-ended, choosing the output with the highest probability (greedy decoding) performs better, challenging the traditional cognition in the LLM field that random sampling improves quality.

The team conducted experiments on multiple VQA benchmark tests, and the results show that greedy decoding not only is not inferior to random sampling strategies but also consistently achieves better performance. The experiments cover various mainstream MLLM architectures and models of different scales, and in all scenarios, the accuracy of greedy decoding is higher than or equal to that of strategies such as temperature sampling, Top-k/Top-p sampling. This indicates that for tasks like VQA that require precise answers, randomness has no substantial benefits and may even reduce performance.

For complex reasoning scenarios, the team proposed the "greedy decoding for reasoning models" method to optimize the processing of intermediate results in multi-step reasoning processes. Experiments show that this improved strategy outperforms standard greedy decoding and traditional random sampling in VQA tasks involving multi-step logical reasoning, which has important guiding significance for building AI systems with visual understanding and logical reasoning capabilities.

Practical Implications: VQA developers can simplify deployment—without tuning sampling hyperparameters, they can directly use greedy decoding to obtain reliable performance, and deterministic outputs are conducive to reproducibility and consistency. Future Directions: Explore optimal decoding strategies for other multimodal tasks; study controllable diversity under greedy decoding; develop task-adaptive decoding strategies.

This study, through theoretical analysis and experimental verification, reveals the superiority of greedy decoding in VQA, corrects inherent biases in the field, and provides practical guidance for multimodal AI deployment. When pursuing improvements in model capabilities, returning to simple and straightforward methods may yield unexpected results.