Bias Detection and Mitigation for Large Language Models: A Post-Processing Debiasing Scheme Based on the Seven-Signal Mixture-of-Experts Architecture

This article introduces an open-source debiasing framework targeting social bias issues in large language models. It corely uses seven-dimensional confidence signal extraction and a mixture-of-experts aggregator to achieve post-processing debiasing without modifying model weights, and has achieved significant results on the BBQ benchmark. This framework addresses the problems of high cost, poor generalization, or over-correction in traditional debiasing methods, providing a new path for AI ethics and fairness research.

With the widespread deployment of LLMs, social bias issues in training data have become prominent. Models tend to reproduce stereotypes when dealing with sensitive attributes, affecting fairness and credibility. Traditional debiasing methods fall into two categories: data cleaning or adversarial training during the training phase, which are costly and hard to generalize; prompt engineering during the inference phase, which easily leads to over-correction and reduced accuracy. How to mitigate bias while maintaining model capabilities has become a core challenge.

The framework is a four-stage pipeline: 1. Multi-prompt reasoning (four types of prompts: standard, debiasing, chain-of-thought, counterfactual replacement); 2. Seven-signal feature extraction (evidence overlap, counterfactual consistency, self-confidence, self-consistency, bias head attention, prompt sensitivity, SAE feature activation); 3. Mixture-of-experts aggregator (four expert modules: vocabulary replaceable, numerically verifiable, cultural context, identity-sensitive; a gating network dynamically assigns weights to output bias probability); 4. Threshold-based override decision (retain original answer if p≥0.5, override to "Unknown" if p<0.5).

In the BBQ benchmark test, the framework maintains high accuracy while significantly reducing bias scores. Cross-model transfer verification: full transfer from Llama-3.1-8B to Gemma-2-9B achieves good results; when transferring to Qwen-2.5-7B, setting the SAE signal to zero still maintains comparable performance. In addition, it shows good generalization ability in zero-shot tests on ImplicitBBQ and OpenBiasBench.

This scheme provides a pluggable post-processing module that can reduce bias risks without modifying the weights of deployed models. For developers: it improves product fairness and compliance without sacrificing performance; for researchers: it provides a systematic bias evaluation tool to guide the design of safe AI.

Current limitations: mainly targeted at English environments, with limited coverage of other languages and cultures; the "Unknown" override strategy may affect user experience. Future directions: expand SAE feature analysis to more model families, develop adaptive threshold mechanisms, and explore hybrid schemes that feed debiasing signals back into fine-tuning.