TrustgameLLM: Do Large Language Models 'Treat People Differently Based on Their Background' When Playing the Trust Game?

This study uses the classic trust game experimental framework to systematically examine whether large language models (LLMs) exhibit differentiated strategies based on social identities such as gender and nationality in interactive decision-making. The results show that LLMs do adjust their cooperative behaviors according to the social identities of virtual opponents, revealing potential bias patterns in training data, which is of great significance for AI fairness research.

When we interact with LLMs like ChatGPT, do we get different responses based on user background differences? The TrustgameLLM project conducts research on this question. The trust game is a classic paradigm in behavioral economics: Player A transfers part of their funds to Player B (the amount is tripled), and Player B decides how much to return. The core is the win-win of trust and reciprocity or the loss from selfishness.

The study has LLMs play one side of the game, and the other side is a virtual human with different gender (male/female) and nationality characteristics (presented only in text, no behavioral differences). Core hypothesis: If LLMs are influenced by identity cues, there will be systematic differences in the amount of investment (level of trust) when facing opponents of different identities.

The results show that LLMs adjust their cooperation strategies: 1. Gender differences: Some models invest more in opponents of specific genders, suggesting gender stereotypes in training data; 2. Nationality bias: Different levels of trust towards opponents of different nationalities, reflecting uneven distribution of national images in training corpora. These biases come from statistical patterns in training data and are not intentionally implanted by developers.

The TrustgameLLM project provides complete code and datasets, with features including: standardized interfaces supporting calls to multiple mainstream LLMs; configurable identity characteristics of virtual players; complete recording of decision-making processes and results; control experiments to distinguish identity effects from random fluctuations, facilitating reproduction and expansion.

The significance of the research goes far beyond academia: LLMs are widely used in scenarios such as customer service, recruitment, and credit. If there are identity biases in simple games, unfair consequences may be amplified in complex scenarios (e.g., biased decisions in loan approvals due to gender/nationality cues).

Future research can be carried out from four aspects: 1. Trace the sources of bias and identify key corpora in training data; 2. Develop debiasing technologies (fine-tuning/inference stages); 3. Establish bias detection benchmarks to improve transparency; 4. Expand cross-cultural research to reveal the cultural specificity of biases.