When AI Processes Public Opinions: Do Large Models Have Systemic Bias Against 'Grassroots Voices'?

Core point: A large-scale controlled experiment on 8 federally available LLMs found that occupation is the only identity signal leading to consistent differential treatment. When the same comment is attributed to a street vendor instead of a financial analyst, the summary loses more original meaning, uses simpler language, and shifts emotional tone. This study focuses on the fairness issue of AI processing public comments in the U.S. federal 'Notice and Comment' mechanism, revealing potential systemic occupational bias in AI systems, which has important implications for the equality of democratic participation.

The 'Notice and Comment' mechanism in the U.S. federal regulatory system is a core channel for citizens to influence government decisions, covering areas such as environmental protection, food safety, and finance. Theoretically, it ensures that everyone's voice is heard equally. However, when federal agencies use large language models (LLMs) to process massive public comments, a key question arises: Can AI systems truly treat all voices equally? Do identity signals affect AI's understanding and summarization of comments?

The research team designed counterfactual experiments: keeping the comment content completely consistent while only changing the commenter's identity attributes (race, gender, occupation) to observe changes in AI summaries. The experimental setup includes: 182 real public comments, 32 identity conditions, 8 federally available LLMs, and over 106,000 summaries. Identity signals are manipulated through signature information: race (typical racial names), gender (names + pronouns), occupation (socioeconomic status indicators such as street vendor vs. financial analyst).

The research results show: 1. Systemic occupational bias exists: When the same comment is attributed to a street vendor, the summary's semantic fidelity decreases, language is simplified, emotional tone shifts, and this is consistent across all models/contexts; 2. Unstable race effect: Differences are driven by specific name tokens, not real racial categories, and model responses vary greatly; 3. No gender effect: There is no systemic difference in summary quality between male and female signatures.

In-depth analysis found: Writing quality affects summary results, but it focuses on substantive arguments rather than surface spelling or grammar errors; The root causes of occupational bias may come from training data: ① Occupation-language association (differences in writing styles across occupations); ② Authority heuristic (models internalize the stereotype that "professional opinions are more reliable"); ③ Audience adaptation mechanism (models adjust output style based on expected audience, which becomes discrimination in government scenarios).

The degree of occupational bias varies significantly among different model providers, meaning that when the government chooses an LLM, it implicitly chooses a level of fairness. Existing federal IT procurement frameworks (such as FedRAMP) focus on security, privacy, usability, and cost in their evaluations, and do not include fairness (consistency in processing for different groups) in their standards, resulting in blind spots.

Policy implications: 1. Socioeconomic status (occupation signals) should be included in AI fairness assessments; 2. Integrate fairness benchmark tests into federal procurement processes (testing during model selection, parallel with other indicators, regular re-evaluation); 3. Consider stripping/anonymizing identity information from public comments to balance context understanding and fairness.

Broader significance: AI bias is multi-dimensional (different impacts from occupation, race, etc.), scenario-sensitive (personalization in entertainment recommendations may become discrimination in government scenarios), and technical solutions have limitations that require institutional guarantees (procurement standards, audits, manual reviews). Conclusion: Technology is not neutral; AI carries biases from training data. Fairness requires deliberate design, continuous monitoring, and improvement to avoid the accumulation of inequalities in democratic participation.