Study on the Impact of Polish Query Errors on the Response Quality of Large Language Models

This study is a master's thesis project that systematically analyzes the impact of errors (spelling, grammar, etc.) in Polish queries on the output quality of large language models. It aims to evaluate the robustness of models in multilingual scenarios and provide empirical evidence for improving the performance of LLMs in non-English languages. The study covers aspects such as background, methodology, key findings, and application value, which will be elaborated in the following floors.

With the global application of large language models (LLMs), the processing performance of non-English languages (especially with error-containing inputs) has become a key issue. Polish is an important European language (with 45 million speakers), but there is a lack of research on model performance in error-input scenarios related to it. Daily user queries often have spelling, grammar, and other errors, which may lead to comprehension deviations in English-core models. Therefore, a special study on Polish is needed.

Construct a dataset containing various types of Polish errors: covering character-level (letter replacement/omission), lexical-level (word form change errors), and syntactic-level (word order issues) errors, generated based on actual error patterns of native speakers, with each error sample paired with a standard control group; select mainstream open-source and commercial models to conduct multi-model comparison experiments, quantifying the impact of different errors on output quality.

1. Differences in error type sensitivity: Character-level errors have limited impact, while Polish-specific grammatical errors (such as case changes) lead to severe comprehension deviations; 2. Non-linear relationship between model size and robustness: Large models perform well in standard tests, but their advantages are weakened when handling specific Polish errors; 3. Limitations of cross-language transfer: The fault tolerance of English-optimized models cannot be directly transferred to Polish.

In product design, input preprocessing can be optimized (such as automatic spelling check/query rewriting) to improve the Polish user experience; when selecting models, refer to the distribution of error types to choose appropriate models or conduct targeted fine-tuning; multilingual strategies need to be optimized for the unique characteristics of each language and cannot copy the experience from English scenarios.

The project is released as open-source, including complete source code and datasets. Components include: dataset generation and processing tools, model evaluation framework, error injection and transformation module, and result analysis visualization scripts, promoting reproducible research and industrial application expansion.

Limitations: The dataset size is limited and does not cover all Polish error patterns; only focuses on text generation tasks. Future directions: Expand to more Slavic languages; study the improvement effect of error correction mechanisms on model performance; explore model fine-tuning methods for multilingual error robustness; build larger-scale multilingual error datasets.