Aqal: The First Urdu Reasoning-Optimized Large Language Model

Aqal is the world's first reasoning-optimized large language model specifically designed for Urdu. Through a three-stage training process (continuous pre-training, supervised fine-tuning, and GRPO reinforcement learning), it significantly improves multi-step reasoning, logical consistency, and the correctness of final answers, filling the gap in high-quality reasoning models for low-resource languages like Urdu.

Urdu is one of the official languages of Pakistan, with over 170 million speakers worldwide. However, as a low-resource language, traditional large language models face issues such as scarce training data, insufficient reasoning capabilities, and poor logical consistency. The Aqal project aims to fill this gap and explore systematic methods to enhance the reasoning performance of Urdu models.

Aqal uses a three-stage training approach:

1. Continuous Pre-training: Builds foundational language capabilities based on multi-domain Urdu corpora;
2. Supervised Fine-tuning: Introduces high-quality reasoning datasets to train the model to follow instructions and perform multi-step reasoning;
3. GRPO Reinforcement Learning: Enhances reasoning quality through policy optimization and reward modeling, addressing the instability issues of traditional reinforcement learning.

Aqal adopts a modular architecture:

• Environment Management: Conda-isolated environment;
• Dependencies: Python 3.10+ and a clear requirements.txt;
• Code Structure: Includes main entry scripts, training modules (e.g., GRPO trainer), and evaluation tools (e.g., reasoning evaluation scripts), facilitating research and development.

The significance of Aqal includes:

• For the Urdu community: Provides support for mother-tongue education, business applications, and cultural heritage preservation;
• Technical Paradigm: The three-stage training process can serve as a reference for the development of models for other low-resource languages, promoting balanced global language technology development.

Aqal, through its systematic training method, builds high-quality reasoning models for low-resource languages and promotes AI democratization. We look forward to more breakthroughs in the Urdu AI field and encourage researchers to focus on the inclusivity and diversity of language technologies.