How Generative AI Literacy Training Enhances Intelligence Analysts' Ability to Identify Real vs. AI-Generated Images

An experimental study involving 32 intelligence analysts shows that after 30 minutes of generative AI literacy training, their accuracy in distinguishing AI-generated images from real photos improved significantly. The research team has made the complete dataset, code, and training materials publicly available, providing a reproducible scientific basis for cultivating AI image identification skills.

With the rapid development of generative AI technology, AI-generated images have reached a level of realism that can easily deceive the human eye. For intelligence analysts, accurately distinguishing between real and AI-generated images directly affects the quality of intelligence and the reliability of decision-making. However, how effective is the human eye at identifying AI-generated images? Can short-term AI literacy training effectively improve this ability? These questions urgently need scientific verification.

The research team from Northwestern University conducted this study, recruiting 32 intelligence analysts to participate. The dataset includes 50 AI-generated images and 47 real photos; the training materials are 30-minute slides explaining the characteristics of generative AI images and identification methods. The experiment used a pre-post comparison design, recording changes in each analyst's judgment accuracy before and after training, while also collecting background information such as participants' experience in image forensics and frequency of exposure to AI-generated images.

The experimental results show that the analysts' identification accuracy improved significantly after training. The team collected 365 comments from analysts, used large language models to code and classify them to build an image artifact classification system; they also extracted image features through computer vision, analyzed visual features related to human identification accuracy, providing a new perspective for understanding the differences between human and machine image recognition.

The study proves that short-term targeted AI literacy training can effectively improve professionals' identification ability, providing a scientific basis for enterprises and governments to design training programs. The research has open-sourced materials such as datasets, code, and analysis scripts, making it easy for other researchers to reproduce the results. The 30-minute training duration is feasible, providing an operable training framework for organizations such as intelligence agencies and news media.

The GitHub repository contains complete reproducible materials (data files, analysis code, training materials, ethical review instructions). Participants' personal information has been hashed to protect privacy while ensuring data availability. Python standard tools were used for analysis, with clear dependency library versions, reducing the threshold for reproduction.

The study provides strong evidence for image literacy education in the era of generative AI, showing that targeted training can improve professionals' ability to identify AI-generated content. Implications for AI governance: Technical literacy education is an important way to address the challenges of generative AI, and can form a defense system together with human-machine collaboration.