GenAI Risk Discourse: A Research Framework for Analyzing Generative AI Ethical Risk Discourse on Social Media Using Large Language Models

GenAI-Risk-Discourse is an open-source academic project that provides complete reproducible materials for identifying and analyzing generative AI ethical risk discourse on social media using large language models (LLMs). By combining LLM technology with traditional discourse analysis methods, the project addresses the limitations of traditional research in handling massive unstructured data, offering a new approach to studying public discourse on AI ethics with both academic and practical value.

Wave of Ethical Discussions in the Generative AI Era

Since the release of ChatGPT at the end of 2022, generative AI has permeated various fields of society, and ethical risks such as copyright issues and misinformation have sparked a wave of discussions on social media. Traditional survey methods are limited by sample size and timeliness, making it difficult to capture public opinion dynamics; while the massive and real-time nature of social media data offers new research possibilities, it also brings challenges in identification and analysis.

Project Background and Academic Value

GenAI-Risk-Discourse was developed by the SYJKim team as open-source reproducible material for related papers. Its academic value includes: filling the gap in empirical analysis of public discourse in AI ethics research; demonstrating the innovative application of LLMs in social science research—using semantic understanding and reasoning capabilities to achieve more detailed discourse identification, going beyond traditional keyword matching or machine learning classification.

Core Question: How to effectively identify and classify generative AI ethical risk discourse on social media?

Mixed-Methods Process

1. Data Collection and Preprocessing: Collect public posts from social media, perform text cleaning, language detection, deduplication, etc.
2. LLM-Assisted Discourse Identification: Adopt few-shot learning prompt engineering to let LLMs judge whether the text involves ethical risk discourse and output confidence scores.
3. Discourse Classification System: Establish a multi-dimensional framework (risk types such as copyright/misinformation, discourse functions such as risk warnings/policy appeals), with manual verification after initial classification by LLMs.
4. Discourse Analysis Framework: Analyze deep features like emotional tendencies, rhetorical strategies, and attribution patterns to understand public discussion modes.

The project provides complete technical implementation:

1. Data Collection Module: Use social media APIs or crawlers, handle rate limits and error retries, and ensure compliance.
2. Preprocessing Pipeline: Modular design covering text encoding, language recognition, tokenization, etc.
3. LLM Interaction Layer: Encapsulate APIs of different models (e.g., GPT, Claude), supporting batch processing, error handling, and caching.
4. Analysis Scripts: Based on the Python ecosystem (pandas, transformers, etc.), implement end-to-end analysis workflows.
5. Visualization Tools: Generate time-series graphs (discussion heat), distribution charts (risk type proportions), network graphs (topic correlations), etc.

Although there are no specific results in the reproducible materials, inferences can be drawn from the methodology:

• Discussion heat fluctuates with major AI product releases or controversial events;
• Users on different platforms have varying focuses on risk types;
• Discourse patterns of professional communities and the general public are significantly different.

The value of these findings for AI governance: helping policymakers prioritize urgent issues, design effective risk communication strategies, and predict social controversies.

Widely applicable methodologies of the project:

• Researchers: Migrate to analysis of other technical ethical issues such as autonomous driving ethics and gene editing;
• Corporate AI Ethics Teams: Monitor public risk perception of products/industries and respond to reputation risks early;
• Policy Researchers: Support evidence-based policy making and understand concerns of different groups;
• Educators: Use as a case in computational social science courses to demonstrate interdisciplinary methodological innovation.

Limitations

1. Social media data has demographic biases and cannot represent the entire public;
2. LLM judgments are affected by prompt design and model selection, requiring uncertainty analysis;
3. Automated methods may miss subtle meanings captured by human analysts, requiring human-machine collaboration.

Future Directions

• Develop more refined classification systems;
• Establish longitudinal tracking mechanisms to observe long-term discourse evolution;
• Explore multimodal analysis (images, videos);
• Build real-time monitoring systems to support risk early warning.