THEMIS: A Parameterized Legal Knowledge Large Language Model Built Specifically for Indian Law

THEMIS is a domain-specific large language model for law fine-tuned specifically for Indian statutory law. It uses a parameterized knowledge architecture to embed legal reasoning capabilities into model weights (instead of relying on retrieval systems). The project demonstrates the feasibility of training professional LLMs in resource-constrained environments and outlines a complete development roadmap from v1 to v4. The code, datasets, etc., are open-sourced under the MIT License.

THEMIS focuses on Indian statutory law and is a parameterized knowledge model (different from retrieval-based Q&A or API wrapping). Its core philosophy is "HECTOR for retrieval, THEMIS for reasoning". Developers believe that legal intelligence needs to understand the logic of provisions, applicable scenarios, and relationships, rather than just searching for provisions.

Base Model and Training Technology

Based on Mistral 7B Instruct v0.3, using LoRA for efficient fine-tuning to balance reasoning ability and computational efficiency (supports training on Kaggle's free T4 GPU).

Training Data

Covers core Indian laws such as BNS2023, IPC1860, BNSS2023, BSA2023.

Features Implemented in v1

• Successful end-to-end training process on Kaggle
• LoRA adapter released on HuggingFace
• Mastered Alpaca instruction format and legal assistant-style responses
• Correctly trained disclaimer behavior

Limitations of v1

• Confusion in identifying BNS2023 abbreviations
• Inaccurate citation of clause numbers (hallucination)
• Insufficient deep legal knowledge (limited to 1939 training pairs)
• Insufficient retention of transition knowledge from IPC to BNS

Root Causes

Mistral's pre-training data does not include BNS2023 (enacted in December 2023), and LoRA fine-tuning only taught the "way to answer" without mastering specific legal content.

Comparison of v1 Parameters and Targets

Parameter	v1 Value	v2 Target
LoRA Rank	8	32
Sequence Length	512	2048
Number of Training Pairs	1939	50000-90000

v2 Roadmap (In Progress)

Targets: 10,000-15,000 training pairs, LoRA rank 16, sequence length 1024; improvement directions include expanding datasets, disambiguating BNS abbreviations, and introducing citation accuracy metrics; success criteria: over 70% of criminal law queries correctly identify BNS and cite clauses accurately.

v3 Targets (Planned)

Targets:50,000-90,000 training pairs, LoRA rank32, sequence length2048; training data covers multiple domains such as criminal law, procedural law, evidence law (total of74,000 pairs); success criteria: citation accuracy>85%, hallucination rate<10%.

The ultimate goal is to integrate THEMIS (parameterized reasoning) and HECTOR (retrieval-augmented):

1. User query → query classifier determines "parameterized or retrieval"
2. THEMIS handles citizen-level Q&A reasoning
3. HECTOR handles in-depth research requiring PDF citations
4. A unified router distributes the query, and the output includes both citations and reasoning.

1. Feasibility of Domain Models: LoRA technology enables training of professional LLMs in resource-constrained environments (free GPUs), providing a reference for vertical domain AI.
2. Importance of Data Scale: The 1,900 pairs in v1 only taught the "way to speak", while 70,000 pairs are needed to "understand the domain", which is a reference for fields like healthcare/finance.
3. Thoughts on Legal AI Architecture: Choosing a parameterized knowledge positioning, emphasizing that legal reasoning requires deep understanding rather than just retrieval.
4. Open-Source Value: Open-sourcing code, datasets, etc., under the MIT License promotes technology dissemination and reuse.

THEMIS is an important exploration in the field of legal AI, proving that building a useful domain model requires sufficient data, a reasonable architecture, honest evaluation, and long-term iteration. Its roadmap from the limitations of v1 to the ambition of v3 provides valuable references for vertical domain AI developers.