LLMT: A Machine Translation Framework Based on Large Language Models

LLMT is an open-source machine translation framework focused on using Large Language Models (LLMs) for high-quality translation. This framework provides a complete translation pipeline covering data preprocessing, model fine-tuning, inference optimization, and evaluation tools. It supports multiple mainstream LLM architectures and training strategies, aiming to address the limitations of traditional neural machine translation in low-resource languages, domain adaptation, and other aspects, and provides a solid technical foundation and community support for developers and researchers.

The field of machine translation has undergone a paradigm shift from statistical machine translation to neural machine translation, and then to translation based on large language models. Traditional neural machine translation models have limitations in low-resource languages, domain adaptation, and context understanding. The emergence of LLMs provides new possibilities to solve these problems. The LLMT project was born to build an LLM framework for translation tasks, making full use of the language understanding and generation capabilities of LLMs, and optimizing for translation scenarios.

LLMT provides an end-to-end solution from data preparation to model deployment, supporting multiple mainstream LLM architectures (Transformer encoder-decoder and decoder-only architectures). Users can start training and inference through simple configuration files without writing a lot of code. At the data processing level, it has built-in tools such as parallel corpus cleaning, sentence alignment, and subword segmentation, and also provides data augmentation functions like back-translation and noise injection to improve performance in low-resource scenarios.

LLMT supports training paradigms such as Supervised Fine-Tuning (SFT), Instruction Fine-Tuning, and Reinforcement Learning from Human Feedback (RLHF). It uses efficient technologies like gradient accumulation, mixed-precision training, and distributed data parallelism, enabling consumer-grade hardware to train large-scale models. Special prompt templates are designed for translation tasks to guide the model to output as a translation expert, clearly specifying the source/target languages and providing context examples, which significantly improves translation quality.

LLMT integrates inference optimization technologies such as model quantization (INT8/INT4), KV cache optimization, batch inference, and speculative decoding, achieving near-real-time translation latency while maintaining quality. It supports model service deployment, providing an inference server based on FastAPI that supports RESTful API and streaming responses, making it easy for users to deploy the model as a translation service and integrate it into existing applications.

LLMT has a built-in comprehensive evaluation system that supports automatic evaluation metrics such as BLEU, chrF++, and COMET. It also provides manual evaluation tools (A/B testing, TER calculation) to help developers understand model performance and identify error patterns. In addition, it supports translation quality estimation, which can evaluate quality without reference translations, helping with quality control in production environments and screening low-quality translations that need manual review.

LLMT is suitable for scenarios such as document translation, real-time dialogue translation, and code comment translation. In the future, it is expected to expand to support multimodal functions like speech translation and image-text translation. For developers and researchers who want to build translation systems using LLMs, LLMT provides a solid technical foundation and active community support.