ChartCynics: A Dual-Path Agent Framework to Uncover Visual Deception in Misleading Charts

ChartCynics achieves an accuracy of 74.43% on misleading chart question-answering tasks, a ~29% improvement over the baseline, through a dual-track mechanism of diagnostic visual path and OCR data path, combined with two-stage training of Oracle-Informed SFT and Deception-Aware GRPO. This framework aims to address the cognitive challenges posed by visual deception and provide new design insights for trustworthy AI systems.

In the era of information explosion, charts have become a primary carrier for data dissemination, but they are also easily used for misleading purposes (e.g., inverted axes, distorted scales, etc.). Existing vision-language models (VLMs) perform well on standard chart understanding tasks, but they have limitations when dealing with misleading charts: their holistic perception strategy makes it difficult to identify local structural anomalies, and they lack an inherent understanding of "visual deception", unable to actively question what they see.

ChartCynics introduces a "skepticism" reasoning paradigm to actively verify chart information. The framework adopts a dual-path architecture:

1. Diagnostic Visual Path: Focuses on high-risk areas (e.g., axes, legends) through strategic ROI cropping to detect visual anomalies;
2. OCR Data Path: Extracts raw values to reconstruct data tables, and verifies by comparing with visual presentations; The outputs of the two paths are cross-validated by the agent's summarizer to ensure conclusion consistency.

ChartCynics uses two-stage training:

1. Oracle-Informed SFT: Injects expert knowledge through reasoning distillation to learn to identify deception patterns and verification methods;
2. Deception-Aware GRPO: Adversarial alignment training that uses complex adversarial examples to enhance robustness, rewarding deception detection and penalizing being misled.

On the misleading chart question-answering benchmarks, ChartCynics achieves accuracies of 74.43% and 64.55%, a ~29% improvement over the Qwen3-VL-8B baseline, and outperforms the current state-of-the-art proprietary models. Error pattern analysis shows that the remaining errors are mainly due to information overload caused by combinations of multiple deception methods.

The core contributions of ChartCynics include:

1. Decoupling of Perception and Verification: Establishes an independent verification channel to enhance resistance to adversarial attacks;
2. Skepticism Design Principle: Treats "question until proven" as the default stance for high-risk AI systems;
3. Inherent Interpretability: The dual-path design makes the decision-making process traceable, allowing it to point out problem areas and supporting evidence.

The application scenarios of ChartCynics include:

• Automated fact-checking and media literacy education;
• Chart verification in financial analysis and business intelligence;
• Assistance in scientific communication and peer review. Future research can explore dynamic chart deception detection, real-time verification of interactive visualizations, and multi-modal joint reasoning.