CLSGen: A Dual-Head Framework Enabling Large Models to Both Predict Probabilities and Explain Reasons

The CLSGen (Classification and Generation) framework addresses the core challenge of balancing probability estimation and explanation generation in large models' classification tasks through a dual-head architecture design and collaborative training strategy. This framework is suitable for high-risk fields such as medical diagnosis and financial risk assessment, enabling AI systems to output reliable probability estimates while generating human-understandable explanations, achieving the goal of 'not only telling us "what it is" but also explaining "why"'.

In high-risk fields like healthcare and finance, developers face two major challenges with large model decision-making: the need to provide both reliable probability estimates and human-understandable explanations, but existing models cannot balance these two. Generative use can output explanations but lacks precise probabilities; discriminative use (e.g., adding a classification head) can output probabilities but loses the ability to generate explanations. Traditional discriminative fine-tuning also leads to catastrophic forgetting (losing general language knowledge) and language collapse (losing explanation generation ability), forcing a choice between the two in the end.

Dual-Head Architecture Design

• Shared Backbone: Keep Transformer layers shared to learn rich language representations.
• Classification Head: A lightweight head outputs classification probabilities to ensure accurate estimation.
• Generation Head: A standard language model head maintains the ability to generate explanations.
• Collaborative Training: Both heads initiate collaborative optimization to balance classification and generation capabilities.

Training Strategy

• Multi-Task Learning: Simultaneously optimize classification loss (to ensure probability accuracy) and generation loss (to maintain explanation ability).
• Dynamic Weights: Adjust loss weights based on training progress—focus on generation ability in the early stage and increase classification weight in the later stage.
• Curriculum Learning: Sort training data by difficulty to enable stable learning from simple to complex.

Data Construction

• Paired Data: Each sample includes input text, true label, and human explanation.
• Explanation Diversity: Collect explanations of various styles to enhance generalization.
• Quality Control: Manual review + automatic checks to filter low-quality explanations.

Technical Details

• Probability Calibration: Temperature scaling adjusts probability sharpness, label smoothing prevents overconfidence, and bucket calibration optimizes estimation.
• Explanation Alignment: Attention alignment focuses on decision-related inputs, consistency loss penalizes label-explanation inconsistency, and contrastive learning distinguishes between supporting and non-supporting evidence.
• Decoding Strategy: Constrained decoding ensures key information, diversity sampling generates different expressions, and length control adapts to scenario needs.

Classification Performance

• AUROC Improvement: Achieves a 2-5% AUROC increase across multiple datasets compared to traditional discriminative fine-tuning.
• F1 Score Improvement: Better F1 score on class-imbalanced datasets, balancing precision and recall.
• Probability Calibration: Predicted probabilities are more consistent with actual frequencies, and the reliability diagram is close to the diagonal.

Explanation Quality

• Label-Explanation Alignment: Explanations are highly consistent with predicted labels (positive explanations support the judgment, negative explanations indicate no evidence).
• Readability: Automatic metrics (perplexity, BLEU) and human evaluation show fluent language and clear logic.
• Information Richness: Includes supporting evidence, confidence, and other information to help understand the decision-making process.

Capability Preservation

• No Catastrophic Forgetting: Performance on language modeling tasks is similar to the original model.
• General Task Transfer: Good performance on general NLP tasks outside of fine-tuning.
• Explanation Diversity: Generates diverse explanations for similar inputs, not template-based repetitions.

Medical Diagnosis Assistance

• Output: Disease probability (e.g., 78% malignant) + diagnostic basis (irregular mass, spiculated edges, etc.).
• Value: Doctors use the probability to decide whether to conduct further examinations and evaluate the AI's credibility through the explanation.

Financial Risk Assessment

• Output: Risk probability (e.g., 23% default) + risk factors (overdue records, debt ratio, etc.).
• Value: Managers make informed decisions by combining the probability and explanation.

Content Moderation

• Output: Violation probability (e.g.,91%) + violation reasons (hate speech, violation of community guidelines).
• Value: Moderators quickly understand the basis, improving efficiency and consistency.

Legal Document Analysis

• Output: Risk level (e.g.,67% high risk) + risk description (ambiguous clauses, unclear liability for breach, etc.).
• Value: Lawyers prioritize high-risk documents and quickly locate issues.

Limitations

• Computational Cost: The dual-head architecture and joint training increase resource overhead.
• Data Requirements: Requires annotated data with explanations, which is costly to obtain.
• Multi-Class Extension: Currently focused on binary classification; multi-class classification needs further research.
• Explanation Quality: Occasionally generates low-quality or misleading explanations.

Future Directions

• Develop more efficient dual-head architectures to reduce computational costs.
• Research weak supervision learning to reduce reliance on complete explanation annotations.
• Extend to complex scenarios like multi-label classification and ordinal regression.
• Combine human feedback to optimize explanation quality.

The CLSGen framework provides an elegant solution to the application challenges of large models in real-world decision-making scenarios, balancing probability estimation and explanation generation through its dual-head architecture and training strategy. Its significance lies not only in technical breakthroughs but also in representing a new idea for AI system design: not giving up general capabilities and interpretability while pursuing task performance. As AI's role in high-risk fields grows, this approach that balances accuracy and transparency will become more important. CLSGen proves that there is no need to choose between 'black-box accuracy' and 'transparent interpretability', and it is an important step toward trustworthy and practical AI.