VersusQ: Using Pairwise Comparison to Free Video Quality Assessment from Dataset Bias

Traditional Video Quality Assessment (VQA) methods rely on absolute score prediction and are susceptible to dataset-specific rating habits. VersusQ proposes a pure pairwise comparison framework, using the relative reasoning ability of large models to predict the magnitude of quality differences, achieving breakthroughs in cross-domain generalization and fine-grained ranking. This article will discuss its background, methodology, experiments, and significance.

Video Quality Assessment is a core issue in multimedia processing, applied in scenarios such as streaming adaptive bitrate, generative model monitoring, and compression algorithm optimization. Although existing Large Multimodal Models (LMMs) have potential, the point-wise supervision paradigm that continues to use absolute score prediction has hidden risks: absolute scores mix real perceptual differences, dataset annotation preferences, rater subjective habits, and score distribution characteristics, leading to poor model generalization (analogous to a house price model learning only statistical rules rather than universal standards).

The key insight of the VersusQ team: relative comparison can eliminate absolute scale calibration bias. When humans compare videos, they focus on perceptual differences (clarity, smoothness, color, etc.), naturally stripping away dataset-specific rating habits. Based on this, VersusQ abandons absolute scores and adopts a pure pairwise comparison framework: input two videos, analyze differences in dimensions such as spatial details, temporal coherence, and color fidelity, output a signed continuous magnitude value (the sign indicates superiority or inferiority, the magnitude indicates the gap), balancing relativity and fine-grained quantification.

Implementation challenge: generating interpretable comparison reasons while outputting precise numerical differences. VersusQ introduces Margin-Coupled GRPO (a reinforcement learning method), jointly optimizing two objectives: 1. Relational reasoning (correctly judge the quality order + generate reasonable comparison explanations); 2. Continuous magnitude regression (output precise numerical differences). This strategy ensures consistency between the reasoning process and numerical output (larger magnitude for obvious differences, smaller for subtle ones).

In evaluations on multiple public VQA benchmarks, VersusQ performs excellently: 1. Cross-domain generalization: when the training and test sets have different sources or annotation standards, the generalization performance is significantly better than traditional methods; 2. Fine-grained ranking: magnitude prediction provides reliable and precise ranking, suitable for scenarios such as video encoding parameter selection; 3. Heterogeneous scenarios: stable performance under test sets mixing different resolutions, content, and distortion types, with strong robustness.

Significance of VersusQ: 1. Data efficiency: pairwise comparison annotations are easier to obtain and more consistent; 2. Interpretability: generating comparison reasons provides transparency; 3. Expansion potential: can be extended to tasks such as image aesthetics, audio quality, and text generation quality assessment. Limitations: the O(n²) computational complexity of pairwise comparison needs optimization, and the handling of extreme quality differences requires further research.

VersusQ shifts VQA from absolute score prediction to pairwise difference reasoning, successfully breaking free from the constraints of dataset bias. This paradigm shift not only improves cross-domain generalization ability but also provides new ideas for the field of multimodal quality assessment: relative differences sometimes better reflect the essence of things.