Maistros: A New Breakthrough in Building Greek Large Language Models via Knowledge Distillation

Maistros is an open-source Greek large language model with 8 billion parameters. It transfers capabilities from large reasoning models via knowledge distillation and is fine-tuned on the newly constructed CulturaQA dataset, achieving state-of-the-art performance on nine Greek question-answering datasets and providing a feasible path for the development of low-resource language models.

Current large language model research mainly focuses on high-resource languages like English, and their performance on low-resource languages such as Greek is often unsatisfactory. This gap stems from the scarcity of training data, especially the lack of high-quality language and culture-specific corpora. At the same time, state-of-the-art reasoning models usually contain hundreds of billions of parameters, and even on high-end multi-GPU systems, they take seconds to complete a single inference. This resource requirement limits their practical deployment in ordinary computing environments.

CulturaQA is one of the key innovations of this project; it is a high-quality Greek question-answering dataset generated by large reasoning models and manually filtered. Unlike existing Greek datasets that are mainly for model evaluation, CulturaQA is specifically designed for model training and optimization, covering rich linguistic phenomena and cultural background knowledge. The dataset is built using a generate-filter-validate pipeline to ensure sample quality and diversity.

Maistros uses knowledge distillation technology to transfer the capabilities of large reasoning models to smaller model architectures. Knowledge distillation allows small models (students) to learn the output distribution of large models (teachers) instead of directly learning original labels, thus effectively capturing the reasoning patterns and knowledge representations of the teacher models. Maistros 8B is built based on an open-source architecture and undergoes two-stage training: first, it learns general capabilities from large reasoning models via knowledge distillation, then it is supervised fine-tuned on the CulturaQA dataset to adapt to Greek-specific tasks. The project provides complete training and reproduction code, as well as a 4-bit quantized version to lower the deployment threshold.

The research team conducted a comprehensive evaluation of Maistros on nine independent Greek question-answering datasets, comparing it with nine large language models of different sizes. The evaluation results show that Maistros 8B achieves state-of-the-art performance on these Greek-specific tasks, significantly outperforming general multilingual models. This achievement proves the value of specialized optimization for low-resource languages and provides a reference paradigm for model development of other low-resource languages.

The significance of the Maistros project lies not only in providing a high-performance Greek model but also in demonstrating a feasible path for the development of low-resource language models. By combining knowledge distillation, high-quality dataset construction, and targeted fine-tuning, the research team has proven that it is possible to build competitive language models even in data-scarce situations. The project's open-source code, models, and datasets provide directly referable technical solutions for other language communities.

Maistros opens up new possibilities for the development of low-resource language AI. With the advancement of multilingual technology, we look forward to seeing more languages have high-quality native language models, narrowing the digital divide and making AI technology benefit a wider range of language communities. The experience of this project shows that the combination of technological innovation and open collaboration is an effective strategy to address low-resource challenges.