C-ReD: A Chinese AI-Generated Text Detection Benchmark for Real-World Scenarios

The research team released the C-ReD (Chinese Real-prompt AI-generated Detection benchmark) to address challenges in Chinese AI-generated text detection, such as insufficient model diversity, data homogenization, limited domain coverage, and language characteristic differences. Built on real-world prompts, it covers diverse models and domains, significantly improving the reliability and generalization ability of Chinese AI-generated text detection, and provides a standardized evaluation platform for related research and applications.

While large language models bring convenience to content creation, issues like AI-generated fake news, ghostwritten papers, and phishing emails have triggered an information trust crisis, making the development of reliable detection technologies crucial. However, the Chinese scenario faces unique challenges: insufficient diversity in existing benchmark models, data homogenization (large gap between templated prompts and real user interactions), limited domain coverage, and poor transfer effect of English methods due to language characteristic differences.

C-ReD is designed around three core principles: Authenticity (based on real-world prompts), Diversity (covering more than ten mainstream Chinese large models such as GPT series, Wenxin Yiyan, Tongyi Qianwen, etc.), and Comprehensiveness (covering more than ten domains including news, social media, academic papers, etc.). The dataset has a leading scale: tens of thousands of real prompts (desensitized), balanced human and AI samples, and coverage of different text lengths.

In-domain detection accuracy exceeds 90% and is stable; cross-model generalization is excellent (maintains high accuracy even in zero-shot scenarios and identifies model family features); cross-dataset generalization is better than dedicated datasets, with strong adaptability to new domains and robustness to variants like Traditional Chinese; in terms of adversarial robustness, it still has detection capabilities for simple post-processed texts.

Prompt collection is multi-source (social media, Q&A platforms, etc.), desensitized, and quality-screened; text generation uses temperature sampling, multi-round interaction, post-processing control (removal of model identifiers), and manual verification; human samples are matched with AI samples in theme/length, with diverse sources and real verification.

Application value: Provides a standardized evaluation platform for researchers, training and testing data for developers, helps platform operators maintain content credibility, and supports educational institutions in protecting academic integrity. Limitations: Timeliness requires continuous updates, insufficient coverage of adversarial samples, no involvement of multi-modal content, and in-depth research on causal mechanisms is needed.

Future work: Establish a continuous dataset update mechanism, develop adversarial robust detection methods, expand multi-modal detection, and conduct in-depth research on AI text fingerprint features. Conclusion: C-ReD is an important step in AI governance, balancing innovation and responsibility. We look forward to the community developing more robust detection methods based on this to promote the construction of a trustworthy AI environment.