LLM-Vocabulary-Insight: In-depth Analysis of Greek Tokenization Capabilities of 50 Large Language Models

This project conducts a comprehensive analysis of the Greek tokenization capabilities of 50 mainstream large language models (LLMs), revealing significant differences in multilingual support among different models and providing data-driven references for selecting LLMs suitable for Greek language processing. The project was developed by constLiakos and released on the GitHub platform on June 5, 2026.

With the widespread global application of large language models (LLMs), the issue of their support for different languages has gradually emerged. As the first step in text processing for models, the tokenizer's vocabulary composition directly determines encoding efficiency and comprehension ability. Greek, with its unique alphabet system and historical heritage, has become an ideal test case for evaluating the multilingual capabilities of LLMs, leading to the creation of the LLM-Vocabulary-Insight tool.

The project evaluates 50 mainstream LLMs (with parameter sizes ranging from 7B to 235B). Core metrics include the total number and proportion of Greek tokens, character coverage and tokenization efficiency, and comparisons with the Latin language family. The combined vocabulary of the 50 models exceeds 7.39 million tokens, of which Greek tokens account for approximately 106,000 (1.43%) and Latin tokens reach 4.758 million (64.37%), directly reflecting the language bias in current LLM vocabularies.

There are order-of-magnitude differences in the level of Greek language support among different models. Top-performing models: ilsp/Meltemi-7B-Instruct-v1.5 (28,162 Greek tokens, 45.89% proportion), ilsp/Llama-Krikri-8B-Instruct (22,212 tokens, 14.88% proportion); Worst-performing models: microsoft/phi-4 (44 tokens, 0.04% proportion), ibm-granite/granite-4.0-tiny-preview (42 tokens, 0.09% proportion); Mainstream general-purpose models have proportions ranging from 0.5% to 2%, such as unsloth/Qwen3.5-27B (1,538 tokens, 0.62%) and unsloth/gemma-3-27b-it (1,409 tokens, 0.54%), etc.

There is no simple positive correlation between total vocabulary size and Greek language support rate. For example, mlx-community/aya-expanse-32b-8bit has a total vocabulary of 255,000 tokens, with Greek tokens accounting for 68.12% (173,699 tokens). However, simply expanding the vocabulary does not guarantee better multilingual support; the key lies in the vocabulary composition strategy and the language distribution of training data.

1. Specialized models vs. general-purpose models: In scenarios involving extensive Greek language processing, specialized models (such as Meltemi and Krikri) outperform general-purpose models; 2. Impact on tokenization efficiency: Models with low Greek token proportions produce longer token sequences during processing, increasing computational costs and potentially affecting long-text comprehension; 3. Selection for multilingual projects: It is necessary to consider the coverage of the target language in the model's vocabulary, rather than just focusing on overall performance benchmarks.

LLM-Vocabulary-Insight uses Greek as an entry point to reveal the issue of language support imbalance in the current LLM ecosystem. Data shows that even the best general-purpose models have a much lower proportion of Greek tokens than Latin tokens. The project's methodology is scalable and can be used to evaluate tokenization support for any language, helping to build language-fair AI systems and promote more inclusive AI applications.