# Traditional Statistical Representation Outperforms Generative AI in Identifying Expert Reviewers

> This article evaluates six professional domain identification methods and finds that TF-IDF successfully identifies experts in the top 25 recommendations at a rate of 79.5%, while GPT-4o mini only achieves 51.5%. This indicates that fine-grained vocabulary is more important than semantic smoothness for distinguishing subfield expertise.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-18T17:59:45.000Z
- 最近活动: 2026-05-19T03:32:31.149Z
- 热度: 128.4
- 关键词: 同行评审, 专家识别, TF-IDF, 生成式AI, 信息检索, 学术出版
- 页面链接: https://www.zingnex.cn/en/forum/thread/ai-ebda2f1c
- Canonical: https://www.zingnex.cn/forum/thread/ai-ebda2f1c
- Markdown 来源: floors_fallback

---

## Traditional Statistical Methods (TF-IDF) Significantly Outperform Generative AI in Expert Reviewer Identification

This article evaluates six professional domain identification methods and finds that TF-IDF identifies experts in the top 25 recommendations at a rate of 79.5%, while GPT-4o mini only achieves 51.5%. The study shows that fine-grained vocabulary is better at distinguishing subfield expertise than semantic smoothness, and traditional statistical methods perform better in this task.

## Background and Challenges of Expert Identification in Peer Review

Peer review is the core of academic publishing, but expert identification faces many challenges: the subdivision of modern scientific fields (e.g., differences between subfields of machine learning and reinforcement learning), increased complexity from interdisciplinary research, outdated classifications for emerging fields, and impacts from regional and language factors. There is an urgent need for automated identification systems.

## Research Design and Evaluation Methods

The study treats expert identification as an information retrieval problem, using data from an international observatory review system as a benchmark (author identity serves as an expert proxy). Six methods are evaluated: TF-IDF (traditional statistics), GPT-4o mini (generative AI), and four others (based on citations, keyword matching, etc.). The core metric is whether the marked expert is included in the top 25 recommendations.

## Core Findings and Analysis of Why TF-IDF Outperforms

The results show that TF-IDF has a success rate of 79.5%, while GPT-4o mini only has 51.5%. The reasons include: 1. Fine-grained vocabulary vs. semantic smoothness: TF-IDF relies on precise word matching to capture subfield differences, while generative AI's semantic smoothness tends to overgeneralize and blur boundaries; 2. Interpretability: TF-IDF is transparent and auditable, while generative AI is a black box; 3. Computational efficiency: TF-IDF has fast preprocessing, low cost, and is scalable.

## Implications for AI Applications in Academic Publishing

1. Not all tasks are suitable for generative AI; tools should be selected based on task requirements; 2. Transparency and interpretability are crucial in professional fields; 3. Future research can explore hybrid methods combining TF-IDF and generative AI to balance precision and semantic understanding.

## Research Limitations and Future Directions

Limitations: The evaluation is mainly in the field of astronomy, and performance in other fields needs to be verified; the lack of literature in emerging fields may affect TF-IDF's effectiveness; cross-language scenarios are not covered. Future directions can expand to more fields, emerging fields, and cross-language identification.