Qwen-RBI-RL: Practice of a Professional Financial Domain Reasoning Model Based on Three-Stage Training

This project introduces Qwen-RBI-RL, a domain-specific model trained on India Reserve Bank (RBI) regulatory documents. It uses a three-stage training process (continuous pre-training, cold-start SFT, GRPO reinforcement learning) based on the Qwen3-4B model, achieving verifiable reasoning capabilities in the financial regulatory domain while remaining efficient for deployment on consumer hardware.

General LLMs excel at common tasks but struggle with professional domains like financial regulation due to unique terminology and reasoning patterns. RBI issues large volumes of complex regulatory documents (circulars, master directions) that are critical for practitioners but hard to manually navigate. Qwen-RBI-RL addresses this by building a model specialized in understanding RBI docs.

The model is based on Alibaba's Qwen3-4B, balancing performance and efficiency. Key configs:

• Base model: Qwen/Qwen3-4B
• Params: ~4B
• Context length: 2048 tokens
• Data type: float16 This 4B scale allows it to carry domain knowledge while running on consumer hardware.

1. Continuous Pre-training (CPT): Absorb RBI domain knowledge using 14,175 filtered RBI docs (13,970 circulars, 222 master directions; 5490M tokens). Trained via QLoRA (rank64, 4-bit) with 1e-5 learning rate for 3 epochs (pure RBI data due to URI issue). 2. Cold-start SFT: Teach chain reasoning with <思考>...</思考> format using 5,003 valid trajectories (from Qwen3-6-27B via vLLM). Trained via QLoRA (rank32) with 2e-5 learning rate for 2 epochs. 3. GRPO Reinforcement Learning: Improve factuality using NLI-based rewards. Uses 3,000 verifiable QA pairs (from RBI master directions). Reward: +1 if answer is蕴含 by source, +0.1 for valid <思考>, -0.2 for no <思考>. Trained via GRPO (750 steps,5e-7 learning rate) to reach avg 0.53 NLI score.

Reward Curve: GRPO training reached peak 0.64 at step360, stabilizing at0.53 final score (vs base model's near-zero on RBI tasks). Examples:

• Q: RBI's Basel III minimum CAR? A:8% (with step-by-step reasoning).
• Q: RBI's LCR requirement? A:100% (explains HQLA coverage for30 days). These show the model's ability to reason step-by-step and give factually accurate answers.

1. Verifiable Reward: Uses NLI蕴含 instead of human preferences, ensuring objective factuality.
2. Data Efficiency: Achieves expert-level performance with limited data (5490M CPT tokens +5k SFT +3k RL samples).
3. Reasoning-Knowledge Separation: Teaches reasoning format first (SFT) then optimizes quality (RL).
4. Transparency: Explicit <思考> blocks make reasoning process interpretable.

Limitations:

• Context length (2048 tokens) may not handle long docs.
• Only supports English RBI docs.
• No real-time update for new regulations. Future:
• Integrate RAG for real-time doc retrieval.
• Expand to other financial regulators/regions.
• Add incremental learning for new docs.

Qwen-RBI-RL demonstrates an effective path from general model to domain expert. It provides a practical tool for RBI doc understanding and a reproducible template for vertical domain models (legal, medical). Its open-source code promotes transparency and collaboration in domain AI research.