QG-CoC: A Question-Guided Caption Chain Method for Multimodal Large Models

QG-CoC is a zero-shot prompting method for multimodal large models. It generates image caption chains via question guidance, helping models achieve more fine-grained perception and reasoning capabilities in multi-image scenarios. This method was proposed by researchers from institutions including the University of California, Los Angeles, and the related paper will be presented at the EMNLP 2025 conference.

In recent years, multimodal large language models (MLLMs) face two core challenges when processing multi-image scenarios: difficulty in achieving fine-grained perception and lack of ability to effectively integrate reasoning across multiple visual inputs. Existing prompting methods mostly focus on single images or limited scenarios, leaving a gap in research on general complex multi-image reasoning.

QG-CoC (Question-Guided Caption Chain) is a general zero-shot prompting method. Its core idea is to generate question-related image descriptions via question guidance to form a caption chain, helping models accurately locate key information, establish cross-image associations, and integrate perception and reasoning processes—differentiating it from traditional unguided image description methods.

The research team tested on multi-image datasets (MMIU, MUIRBench) and single-image datasets (MMBench, ScienceQA, MMMU), covering closed-source models (GPT-4o, Gemini-1.5-Flash) and open-source models (LLaVA-OneVision-7B, etc.). The results show that this method performs excellently in various tasks, especially with significant improvements in complex multi-image reasoning scenarios.

The official implementation of QG-CoC has been open-sourced, providing a complete evaluation process. Closed-source models require configuration of OpenAI/Gemini API keys, while open-source models have corresponding environment configuration files. The usage process consists of three steps: generating image descriptions, integrating reasoning processes, and benchmark evaluation. Batch run scripts are also provided to facilitate reproduction.

QG-CoC reveals that prompt engineering should use question information to guide model attention, which has reference value for the design of visual-language interaction schemes. For developers, the zero-shot feature allows plug-and-play use without additional annotation or fine-tuning costs, making it suitable for rapid prototype verification and scenario adaptation.

QG-CoC effectively solves the problem of multi-image perception and reasoning for multimodal large models, providing new ideas for the field of visual-language reasoning. In the future, it is expected to expand to more complex multimodal scenarios such as video understanding and document analysis, promoting the development of multimodal intelligence.