Multi-Agent Complaint System: A LangGraph-Based Automated Financial Complaint Handling System

This project is a multi-agent AI system that uses LangGraph workflows and multiple specialized agents to automate the entire process of handling financial consumer complaints (classification, root cause analysis, routing, solution generation). It addresses issues in traditional manual processing such as time-consuming tasks, classification errors, response delays, and compliance risks, providing financial institutions with multi-dimensional value including efficiency improvement, quality assurance, and compliance guarantee.

The traditional manual approach to handling customer complaints in the financial service industry is resource-intensive and prone to issues caused by human factors. This system is positioned as an enterprise-aware complaint operating system. Through structured processes via LangGraph workflows, it integrates the company's knowledge layer (classification system, severity levels, policies, routing, control candidates) to achieve a complete closed loop from complaint intake to final routing, balancing standardization and flexibility.

Ten-Stage Processing Pipeline

1. Complaint Intake: Receive and standardize raw data
2. Company Knowledge Layer: Dynamically build knowledge such as classification, policies, routing, etc.
3. Classification Mapping and Validation: LLM maps categories and verifies confidence levels
4. Risk Assessment: Calculate risk scores to trigger escalation
5. Root Cause Inference: Analyze the root cause of the problem
6. Solution Planning: Generate personalized recommendations
7. Compliance Review: Check compliance with regulations and policies
8. Quality Gating: Ensure complete fields and logical consistency
9. Routing Decision: Determine the processing path
10. Case Management: Maintain the lifecycle

Multi-Agent Collaboration

The system includes agents for intake, classification, risk, root cause, solution, compliance, review, routing, etc., which collaborate by sharing information through LangGraph state management.

Adopts Retrieval-Augmented Generation (RAG) architecture:

• Vector Database: PostgreSQL + pgvector to store historical complaint vectors, supporting similar case retrieval
• Embedding Model: Defaults to Hugging Face BAAI/bge-small-en-v1.5, configurable with OpenAI API
• Knowledge Retrieval: Reference similar historical cases to improve accuracy
• Data Ingestion: Supports stratified sampling of CFPB public datasets to build the knowledge base

Deployment Process

1. Environment Preparation: Python3.11+, FastAPI
2. Configuration Variables: Set OPENAI_API_KEY and DATABASE_URL
3. Start PostgreSQL (with pgvector): docker compose up -d
4. Ingest Data (Optional): python -m app.retrieval.ingest --sample 50000
5. Start API: uvicorn main:app --reload
6. Test Complaint: Submit JSON request via curl

Evaluation Framework

1. Place labeled test data in the specified directory
2. Run Evaluation Script: python -m app.evals.run_evals
3. View metrics such as classification accuracy and confusion matrix

• Efficiency Improvement: Response time reduced from days to minutes
• Quality Assurance: Standardized processes ensure consistent and professional handling
• Compliance Guarantee: Built-in reviews reduce regulatory risks
• Insight Generation: Root cause analysis identifies systemic issues
• Cost Optimization: Reduce reliance on manual labor and focus on high-value cases

This system demonstrates the potential of multi-agent architecture in automating complex business processes, achieving intelligent and controllable handling through specialized subtasks and structured orchestration. As LLM and RAG technologies mature, it will be applied in more scenarios. The open-source implementation provides a reference for other organizations and is worth researching and learning from by financial institutions.