Legal AI Assistant: A Legal Intelligent Q&A System Based on Agentic RAG

Legal AI Assistant is an intelligent Q&A system for legal professionals. It integrates the LangGraph agent architecture, FAISS vector retrieval, and the Llama 3.3 70B large language model to address the pain points of traditional general AI in legal scenarios—such as being prone to hallucinations and having insufficient accuracy. As a graduation project for ITI's Generative AI course, it demonstrates a complete tech stack from prompt engineering to agent evaluation, providing legal practitioners with a practical intelligent research assistant.

In legal practice, lawyers and law students need to quickly retrieve case precedents and analyze contract clauses. However, general AI lacks in-depth understanding of specific legal documents and is prone to hallucinations or inaccurate suggestions. This project aims to address this pain point by building a truly usable legal intelligent assistant through a complete tech stack.

Core Components: Llama 3.3 70B (Groq API), BAAI/bge-base-en-v1.5 embedding model, FAISS vector database, LangGraph agent framework, PyMuPDF+LangChain document loading, Gradio UI. Workflow: User query → Input cleaning → Vectorization → FAISS retrieves Top4 documents → Sufficiency check → Generate structured answer if information is sufficient; expand query or mark as an out-of-knowledge-base question if insufficient.

Covers 10 typical legal scenarios (force majeure clauses, non-compete agreements, etc.). Data source strategy: real case precedents (CourtListener), real contracts (SEC EDGAR/LawInsider), synthetic documents (to supplement topics lacking public clean PDFs)—balancing authenticity and comprehensive coverage.

Performance Metrics: Faithfulness 0.63/1.0, task success rate 100%, average response latency 1.84 seconds, total API cost $0.004, hallucination marks 2/10 (correctly identifies out-of-knowledge-base questions). Key Findings: Intellectual property ownership and SaaS auto-renewal issues scored low because the knowledge base has no relevant documents. The system can honestly admit its limitations, reflecting responsible design.

Technical Security: API keys stored in Colab Secrets, input cleaning to prevent prompt injection, mandatory addition of lawyer disclaimer. Data Compliance: Only uses public domain/synthetic documents, no real client data, and sources are traceable. Usage Boundaries: For research assistance only; legal decisions require consultation with a licensed lawyer.

1. Value of Agentic RAG: Independently judges information sufficiency and proactively expands queries to improve answer quality; 2. Cost-effectiveness: Completes complex scenario tasks with low API costs; 3. Evaluation-driven development: LLM-based faithfulness scoring helps iteration; 4. Knowledge boundary management: Honestly marks unanswerable questions to avoid hallucinations.

Applicable Scenarios: Legal student case study assistance, lawyer contract clause retrieval, preliminary legal knowledge screening, legal education demonstration. Current Limitations: Small knowledge base size (only 10 topics), only supports English documents, relies on Google Colab environment, insufficient support for complex multi-hop reasoning.