# Qwen-RBI-RL: Practice of a Professional Financial Domain Reasoning Model Based on Three-Stage Training

> A domain expert model trained on Reserve Bank of India (RBI) regulatory documents, adopting a three-stage process of continuous pre-training, cold-start SFT, and GRPO reinforcement learning, achieving verifiable reasoning capabilities at the 4B parameter scale.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-25T07:12:17.000Z
- 最近活动: 2026-05-25T07:23:49.530Z
- 热度: 159.8
- 关键词: 领域专家模型, RBI, GRPO, 强化学习, Qwen3, 持续预训练, NLI蕴含, 金融监管AI
- 页面链接: https://www.zingnex.cn/en/forum/thread/qwen-rbi-rl
- Canonical: https://www.zingnex.cn/forum/thread/qwen-rbi-rl
- Markdown 来源: floors_fallback

---

## Qwen-RBI-RL: A Domain Expert Model for RBI Regulatory Docs with Three-Stage Training

This project introduces Qwen-RBI-RL, a domain-specific model trained on India Reserve Bank (RBI) regulatory documents. It uses a three-stage training process (continuous pre-training, cold-start SFT, GRPO reinforcement learning) based on the Qwen3-4B model, achieving verifiable reasoning capabilities in the financial regulatory domain while remaining efficient for deployment on consumer hardware.

## Background: The Need for Domain-Specific Models in Financial Regulation

General LLMs excel at common tasks but struggle with professional domains like financial regulation due to unique terminology and reasoning patterns. RBI issues large volumes of complex regulatory documents (circulars, master directions) that are critical for practitioners but hard to manually navigate. Qwen-RBI-RL addresses this by building a model specialized in understanding RBI docs.

## Model Base & Architecture Configuration

The model is based on Alibaba's Qwen3-4B, balancing performance and efficiency. Key configs:
- Base model: Qwen/Qwen3-4B
- Params: ~4B
- Context length: 2048 tokens
- Data type: float16
This 4B scale allows it to carry domain knowledge while running on consumer hardware.

## Three-Stage Training Process

**1. Continuous Pre-training (CPT):** Absorb RBI domain knowledge using 14,175 filtered RBI docs (13,970 circulars, 222 master directions; 5490M tokens). Trained via QLoRA (rank64, 4-bit) with 1e-5 learning rate for 3 epochs (pure RBI data due to URI issue).
**2. Cold-start SFT:** Teach chain reasoning with `<思考>...</思考>` format using 5,003 valid trajectories (from Qwen3-6-27B via vLLM). Trained via QLoRA (rank32) with 2e-5 learning rate for 2 epochs.
**3. GRPO Reinforcement Learning:** Improve factuality using NLI-based rewards. Uses 3,000 verifiable QA pairs (from RBI master directions). Reward: +1 if answer is蕴含 by source, +0.1 for valid `<思考>`, -0.2 for no `<思考>`. Trained via GRPO (750 steps,5e-7 learning rate) to reach avg 0.53 NLI score.

## Training Results & Performance

**Reward Curve:** GRPO training reached peak 0.64 at step360, stabilizing at0.53 final score (vs base model's near-zero on RBI tasks).
**Examples:**
- Q: RBI's Basel III minimum CAR? A:8% (with step-by-step reasoning).
- Q: RBI's LCR requirement? A:100% (explains HQLA coverage for30 days).
These show the model's ability to reason step-by-step and give factually accurate answers.

## Key Technical Highlights & Innovations

1. **Verifiable Reward:** Uses NLI蕴含 instead of human preferences, ensuring objective factuality.
2. **Data Efficiency:** Achieves expert-level performance with limited data (5490M CPT tokens +5k SFT +3k RL samples).
3. **Reasoning-Knowledge Separation:** Teaches reasoning format first (SFT) then optimizes quality (RL).
4. **Transparency:** Explicit `<思考>` blocks make reasoning process interpretable.

## Limitations & Future Directions

**Limitations:**
- Context length (2048 tokens) may not handle long docs.
- Only supports English RBI docs.
- No real-time update for new regulations.
**Future:**
- Integrate RAG for real-time doc retrieval.
- Expand to other financial regulators/regions.
- Add incremental learning for new docs.

## Conclusion & Application Value

Qwen-RBI-RL demonstrates an effective path from general model to domain expert. It provides a practical tool for RBI doc understanding and a reproducible template for vertical domain models (legal, medical). Its open-source code promotes transparency and collaboration in domain AI research.