DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification

DeltaRubric is a reward modeling method proposed to address the challenges of multimodal AI evaluation. Its core lies in the joint planning and verification mechanism, aiming to build a reliable, comprehensive, and interpretable evaluation system, providing new insights for large model training and evaluation.

Large language models and multimodal AI are developing rapidly, but traditional reward models struggle to meet the evaluation needs of complex multimodal tasks (e.g., single-modal focus, simple scoring mechanisms). DeltaRubric was created to address this challenge.

DeltaRubric divides reward modeling into two phases:

• Planning Phase: Dynamically generates targeted evaluation criteria (e.g., dimensions like accuracy and completeness of image descriptions);
• Verification Phase: Conducts item-by-item checks based on the criteria to form structured judgments, with an interpretable process.

Through unified multimodal representation learning, DeltaRubric can seamlessly handle cross-modal information (e.g., alignment of text prompts and image features). Application scenarios include image description, visual question answering, multimodal dialogue, etc.

It adopts a modular design, extending multimodal encoders and cross-modal attention mechanisms based on large language models; training may use reinforcement learning/contrastive learning, combined with human preference data to optimize evaluation results.

The value of DeltaRubric:

1. Provides a new paradigm for reward modeling and enhances interpretability;
2. Establishes new benchmarks for multimodal evaluation;
3. Provides accurate reward signals for model training, facilitating reinforcement learning improvements.

Future improvement directions:

• Fine-grained evaluation dimensions;
• Real-time evaluation capabilities;
• Expansion to more modalities such as 3D scenes;
• Continuous optimization by closely integrating human feedback.

DeltaRubric provides a new solution for multimodal AI evaluation through the joint planning and verification mechanism. Its interpretability and structured design support the trustworthy development of AI, making it a research direction worth paying attention to.