Learning from Human Labeling Variability: Capturing Personalized Explanatory Behavior via Cross-Annotator Preference Optimization

This paper proposes the Cross-Annotator Preference Optimization (CAPO) method, aiming to enable large language models (LLMs) to learn and replicate the label-explanation behavior patterns of specific annotators. The core finding of the study is that Human Labeling Variability (HLV) can serve as a stable signal to help models understand annotators' personalized reasoning preferences.

Traditional views regard HLV in natural language processing annotation as noise, but recent studies suggest it reflects reasonable perspective/preference differences among annotators. Free-text explanations provide a window to understand HLV, revealing the reasoning processes and preferences behind annotators' label choices. Core question: Can LLMs learn and replicate the specific label-explanation behaviors of annotators?

The study selected two sentence-pair tasks: Natural Language Inference (NLI, judging logical relationships) and Paraphrase Identification (judging whether sentences have the same meaning). Each task was annotated by 4 different annotators to ensure sufficient data for analyzing individual difference patterns.

Three methods are compared:

1. Prompting method: Directly describe the annotator's style, with limited and unstable performance;
2. Supervised Fine-Tuning (SFT): Fine-tune using data from specific annotators, with better performance than prompting;
3. CAPO (Cross-Annotator Preference Optimization): Learn unique patterns by comparing responses from the target annotator with those from others. CAPO technical details: Construct responses from the target annotator as positive examples and others as negative examples; apply preference optimization techniques; balance label consistency and explanation quality.

Experimental results:

• Prompting method: Baseline performance is limited, and individual pattern capture is unstable;
• SFT method: Significantly better than prompting, effectively learning annotator-specific behaviors;
• CAPO method: Further improvement over SFT, achieving the best results in multiple dimensions, and generalizing to new inputs (not simple memorization, but transferable style representation).

Conclusion: It is proven that HLV can be learned as a stable signal for annotator-specific label-explanation behaviors. Application prospects include:

• Personalized model services: Matching specific user/scenario preferences;
• Scalable explanatory annotation: Learning explanation styles based on history;
• Improvement of annotation quality: Reducing unnecessary disagreements while preserving perspective diversity;
• Optimization of human-machine collaboration: Designing better auxiliary annotation systems.

Limitations: Only covers two tasks, limited amount of annotator data, and model representation interpretability needs improvement. Future directions: Expand to more tasks/domains, combine active learning for efficient data collection, and develop better evaluation metrics to measure the quality of annotator modeling.