Ragly: A SaaS Intelligent Customer Service Platform Based on RAG Architecture — Integration Practice of Enterprise Knowledge and Large Language Models

This article analyzes the Ragly project—a SaaS intelligent customer service platform based on Retrieval-Augmented Generation (RAG) technology—showing how it integrates large language models with enterprise internal documents to solve the problem of accurate knowledge Q&A in customer service scenarios. Key coverage includes: enterprise knowledge management pain points and RAG solutions, technical architecture breakdown, SaaS multi-tenant design, customer service scenario challenges, implementation best practices, and future evolution directions.

Core Dilemmas of Enterprise Knowledge Management

General large models have smooth conversations but lack enterprise internal proprietary information (product manuals, processes, customer data, etc.), and sensitive data cannot be directly used to train public models.

Value of RAG Architecture

Without modifying LLM parameters, it dynamically retrieves enterprise documents and injects them into prompts during inference, balancing answer accuracy, timeliness, and data privacy.

Positioning of Ragly

Productize the RAG architecture into a SaaS platform, focusing on customer service/helpdesk scenarios and providing out-of-the-box intelligent Q&A services.

Document Processing and Vectorization

Parse multi-format documents such as PDFs, Word files, and web pages, split them into text segments, and convert them into semantic vectors via embedding models.

Vector Database and Indexing

Store vectors and implement Approximate Nearest Neighbor (ANN) search to quickly match query-related document segments, which affects retrieval speed and accuracy.

Retrieval and Generation Collaboration

Sort and compress retrieval results, guide model generation with prompt engineering, and filter low-quality content by evaluating relevance.

Tenant Isolation

Achieve data isolation (documents, conversations, permissions) via tenant IDs to ensure no knowledge sharing between enterprises, covering database, API, cache, and other layers.

Resource Scheduling and Cost Optimization

Intelligent model call management: use small models for common questions, large models for complex ones; batch process requests, cache results, and dynamically scale up/down.

Configurability and Customization

Provide interfaces to customize answer styles, knowledge base scope, human transfer rules, multi-language support, etc., without modifying code.

Accuracy Assurance

Ensure information is up-to-date and authoritative; be honest when uncertain; provide source document references to trace origins to avoid the risk of incorrect answers.

Context Coherence

Maintain conversation state, handle anaphora resolution (e.g., "this product"), and combine conversation history during retrieval.

Human-Machine Collaboration Mechanism

When confidence is low, users request human assistance, or sensitive operations are involved, seamlessly transfer to human customer service and pass the complete conversation context.

Data Quality is the Foundation

Documents need to have clear structure, accurate information, and timely updates; messy structure/outdated information will seriously affect effectiveness.

Joint Optimization of Retrieval and Generation

Monitor recall/precision (retrieval), relevance/hallucination rate (generation), and optimize both links collaboratively.

Continuous Iteration and Feedback Loop

Establish user feedback mechanisms (satisfaction, human transfer signals) to optimize strategies; dynamically sync knowledge bases (new products, policy updates).

• Multimodal RAG: Support processing of multimodal content such as images, videos, and voice
• Agentization: Upgrade to an agent that can perform operations (place orders, make appointments, modify accounts)
• Personalization: Provide differentiated services based on user historical behavior/preferences
• Real-time Learning: Quickly adapt to new features and common questions from conversations without retraining Ragly demonstrates the potential of RAG architecture in enterprise-level applications, making LLMs controllable and trustworthy intelligent assistants.