Study on Bias in Large Language Model Peer Review: A Technical Examination of Academic Fairness

The oamin-ai team conducted a systematic study on dimensions such as institutional prestige bias and racial bias of large language models (LLMs) in academic peer review through controlled experiments, revealing potential risks of AI-assisted academic evaluation systems and proposing improvement directions. The study emphasizes that technological progress must balance efficiency and fairness, providing important references for AI ethics and academic justice.

Academic peer review is a core mechanism for maintaining research quality, but the surge in submissions and shortage of reviewers have prompted journals to explore large language model (LLM)-assisted review. However, the question of whether AI systems perpetuate or amplify human social biases has not been fully verified. The oamin-ai team launched the llm-peer-review project, focusing on institutional prestige bias and racial bias, and quantitatively evaluating the fairness performance of mainstream LLMs in simulated review tasks through controlled experiments.

The study uses the controlled variable method and designs multiple experimental scenarios:

• Prestige bias experiment: The same paper is labeled as from top universities (Harvard, MIT, Stanford) vs. ordinary institutions, comparing score differences;
• Racial bias experiment: Adjusting the cultural characteristics of authors' names (Western vs. Asian vs. African names) to detect evaluation deviations;
• Income bias experiment: Exploring the model's attitude differences towards research results from regions with different economic backgrounds. All experimental data are standardized to ensure comparability and statistical significance.

The project uses modular code organization:

• The experiments/ directory contains three core experimental modules: ethnicity-bias, prestige-bias, and income-bias;
• The data/ directory stores processed_papers (processed paper samples) and metadata; All code follows the MIT open-source license, supporting free use and improvement by the academic community, enhancing result credibility and providing an extensible foundation for subsequent research.

The research framework reveals important insights:

1. LLM biases stem from implicit social structural biases in training data, not explicit programming instructions;
2. Academic review scenarios are sensitive to biases, and small systematic deviations will accumulate over time to produce significant structural impacts;
3. Controlled variable experiments provide an actionable paradigm for AI fairness evaluation and specific basis for policy formulation.

Reference value for journals and academic institutions:

1. Prudent deployment: Do not use LLM review results as the main decision-making basis until bias issues are fully mitigated;
2. Continuous monitoring: Deploying AI-assisted tools requires establishing bias detection mechanisms and regularly evaluating fairness;
3. Human-machine collaboration: Position AI as an auxiliary tool and retain the final judgment right of human reviewers;
4. Transparency and openness: Journals using AI-assisted review should disclose the facts to authors to maintain academic integrity.

Current limitations: Only focusing on text-level biases, not involving multimodal scenarios; experiments are based on simulated environments, which have gaps with the complexity of real reviews. Future expansion directions:

• Expand the model coverage to compare more commercial and open-source LLMs;
• Introduce real review data to verify the external validity of conclusions;
• Develop bias mitigation technologies (fine-tuning, prompt engineering, etc.);
• Extend to other academic evaluation scenarios such as fund review and award selection.

The llm-peer-review project by oamin-ai provides a concrete case for AI ethics research, reminding us that technological progress cannot be separated from value scrutiny, and efficiency improvement cannot come at the cost of fairness. In today's era where AI permeates the academic evaluation system, such research is of irreplaceable significance for ensuring technology for good and maintaining academic justice.