SCICON: Eliminating Selection Bias in Multiple-Choice Questions for Scientific Charts

Understanding scientific charts is a challenge in the field of multimodal AI. The prior guidance from multiple-choice options themselves can easily lead models to ignore image evidence and produce biases. The research team proposes the SCICON decoding method, which eliminates selection-induced bias by comparing model outputs under image and non-image conditions, thereby improving the accuracy of scientific chart question answering.

Selection-induced bias refers to the phenomenon where models over-rely on the semantic information of multiple-choice options (such as scientific terms) instead of making judgments based on image content, leading to "correct answers by guessing without looking at the chart". In the scientific field, this bias can cause models to ignore precise quantitative information or subtle structures in charts, resulting in incorrect answers.

SCICON is a decoding method that requires no additional training. Its core idea is to subtract the answer score under pure text conditions from the score under image conditions to separate the contributions of text prior and visual evidence. Unlike existing contrastive decoding methods, SCICON directly compares the presence and absence of image conditions instead of perturbing the input.

SCICON stably improves accuracy across three scientific chart question-answering benchmarks (including statistical charts, microscope images, etc.) and three model architectures; it does not rely on specific model architectures and has universality; it requires two forward passes during inference (about twice the overhead), but this is acceptable in scenarios with high accuracy requirements.

SCICON reveals the problem of modal information conflict in multimodal tasks; it demonstrates the "training-free" optimization idea (improving performance through decoding strategies during inference); it emphasizes the importance of task-specific optimization—strategies designed for the structure of multiple-choice questions can unlock greater performance potential.

Application Scenarios: Science education (intelligent tutoring systems), scientific research assistance (literature reading assistants). Limitations: Only applicable to multiple-choice question scenarios and not directly suitable for open-ended question-answering/generation tasks; the computational overhead of two forward passes may affect real-time response scenarios.