The Masked Linguistic Advantage: The Hidden Ability of Large Models to Access Cultural Knowledge via Local Languages

Original Paper Information:

• Original Authors: arXiv authors
• Source: arXiv
• Original Title: The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs
• Link: http://arxiv.org/abs/2606.07422v1
• Publication Time: 2026-06-05T16:16:59Z

Core Viewpoint: When LLMs answer culture-related questions using local languages, their surface performance seems inferior to English. However, after controlling for differences in language proficiency, local languages can better activate the model's cultural knowledge—this advantage is masked by the gap in language ability.

Apparent Contradictions

Intuitively, as the language with the most abundant training data, English is considered the 'universal key' to accessing model knowledge. However, the paper finds that local languages have a hidden advantage in accessing cultural knowledge, which is masked by the model's superior English proficiency.

Limitations of Existing Evaluations

1. Template-based Question Bias: Parallel-translated questions may lose the context and implicit meaning of cultural concepts in local languages.
2. Accuracy Metric Confusion: Raw accuracy conflates language ability (the ability to understand/generate a specific language) with knowledge acquisition (the ability to access cultural knowledge), leading to misjudgment.

2×2 Cross Design

• Question Type: Culture-irrelevant (general questions), culture-specific (requiring specific cultural knowledge)
• Query Language: English, local language

Item Response Theory (IRT) Model

A shared 1PL model is adopted, which can separate model ability and question difficulty, compare performance on a unified scale, control question noise, and thus separate the effects of language ability and cultural knowledge acquisition.

Surface Advantage of English

In raw accuracy, English significantly outperforms local languages on culture-irrelevant questions, which aligns with expectations from training data distribution.

Hidden Advantage Emerges

After controlling for language ability differences, local languages show a positive advantage in cultural knowledge acquisition (for almost all region-model combinations), meaning local languages can better activate relevant cultural knowledge.

Model and Regional Differences

• Cutting-edge models: More obvious local language advantages
• Regionally aligned/language-adapted models: Stronger local language advantages, verifying the importance of language-culture connections.

1. Language-Culture Connection: Language is the carrier and organizational method of cultural knowledge; local languages can activate the inherent language-culture connections, making it easier to access relevant knowledge.
2. Patterns in Training Data: Pre-training learns language-culture co-occurrence patterns (e.g., Chinese culture associated with Chinese language), so queries in related languages easily activate knowledge.
3. Double-Edged Sword of English Proficiency: Strong English comprehension ability, but may not effectively activate cultural knowledge associated with local languages.

Implications for Model Evaluation

• Multilingual evaluation needs to control for language ability differences, translation quality, and cultural content presentation methods.
• Draw on psychometric methods (e.g., IRT) to separate ability dimensions.

Implications for Model Development

• Emphasize high-quality multilingual training data.
• Explore 'language-culture alignment training' to strengthen connections.
• New ideas for prompt engineering: Use local languages for cultural questions, multilingual chain-of-thought strategies.

Limitations

• Covers 13 regions, not including more languages and cultures.
• Focuses on factual cultural knowledge; limited discussion on implicit cultural understanding (e.g., humor, values).
• Static testing, lacks research on dynamic interaction scenarios.
• The neural mechanism of language-culture connections is unclear.

Future Directions

• Expand to more languages and cultures to verify the universality of results.
• Study implicit cultural understanding.
• Explore language-culture relationships in dynamic interaction scenarios.
• Use techniques like probes and attention visualization to study mechanisms.

Reflections on AI Fairness

• Over-reliance on English may systematically underestimate the quality of non-English cultural content, affecting AI fairness and inclusivity.
• Need to develop truly multilingual AI: not only able to speak multiple languages but also fully access relevant cultural knowledge in each language.
• Product design needs to consider cultural sensitivity (default language, switching strategies, etc.).

Conclusion

This study reveals the hidden advantage of local languages in accessing cultural knowledge, providing guidance for model evaluation, development, and application. When evaluating LLM usage, we need to go beyond surface accuracy, understand the complex relationships between language, culture, and knowledge, to develop AI systems that serve the world's diverse cultures.