Multi-Lingual Chain-of-Thought Faithfulness Study: Exploring the Reliability of Cross-Lingual Reasoning in Small Models

This study focuses on the faithfulness issue of small multilingual-first models during chain-of-thought (CoT) reasoning, exploring whether the reasoning process truly affects the final answer, and examining key dimensions such as cross-lingual consistency, causal impact of reasoning, and language alignment. The research aims to fill the gap in the field of cross-lingual CoT faithfulness for small models and provide references for building trustworthy and fair multilingual AI systems.

With the global popularity of large language models, multilingual capability has become an important evaluation metric. However, most studies focus on reasoning ability in English or single-language scenarios, and the CoT faithfulness of small multilingual models in cross-lingual environments has received limited attention. CoT prompting technology improves complex problem-solving ability, but core questions remain: Does the reasoning process truly drive the answer? Extending to multilingual scenarios, is the model's reasoning consistent across different languages? Are there systematic biases in cross-lingual CoT?

To quantitatively evaluate CoT faithfulness, multiple strategies are adopted:

1. Intervention Experiment Design: Modify key intermediate conclusions in the chain of thought and observe answer changes to test causal relationships;
2. Cross-Lingual Comparative Analysis: Select multilingual question pairs that are semantically equivalent but grammatically distinct, and compare the consistency of reasoning logical structures and conclusions;
3. Faithfulness Scoring Mechanism: Establish a multi-dimensional scoring system to evaluate logical coherence, step completeness, hypothesis clarity, conclusion derivability, etc.

Research findings:

1. Unique Challenges of Small Models: Parameter limitations lead to unstable cross-lingual semantic mapping, making reasoning prone to inconsistency;
2. Asymmetric Language Impact: Reasoning in high-resource languages (e.g., English, Chinese) is more detailed and confident, while reasoning in low-resource languages is more concise and conservative, exacerbating differences in user experience;
3. Task Dependence of Faithfulness: In structured tasks (mathematics, logic puzzles), the causal link between reasoning and answers is tight, while in open-ended tasks, the connection is loose.

Application implications:

1. Model Selection and Deployment: In critical decision-making scenarios, prioritize target-language fine-tuned models or larger-scale models;
2. Prompt Engineering Optimization: For low-resource languages, performance can be improved by explicitly requiring "step-by-step thinking" or providing example reasoning chains;
3. Evaluation Framework Improvement: Advocate causal intervention and cross-lingual comparison methods to supplement the deficiencies of traditional accuracy metrics.

Current research limitations: Focuses on classification and reasoning tasks; CoT faithfulness in generative tasks (summarization, translation) remains to be explored; model architecture evolution may change the CoT mechanism. Future directions: Expand to more low-resource languages to build cross-lingual faithfulness maps; explore fine-tuning strategies to improve reasoning consistency; develop automatic diagnostic tools for unfaithful reasoning.