function-agent-rag: A Multi-Agent Collaborative PDF Q&A System

This article introduces an open-source multi-agent PDF Q&A system called function-agent-rag, built on the LlamaIndex framework. The system innovatively adopts a dual-retriever architecture (BM25 + vector retrieval), a property graph indexing mechanism, and a dual-agent workflow of researcher and writer. It aims to solve the hallucination problem in traditional document Q&A and provide precise, evidence-based answers. Its application scenarios cover multiple fields such as enterprise knowledge bases, academic research, legal document analysis, and medical literature querying.

In the era of information explosion, traditional keyword search cannot meet complex query needs; Q&A systems based on large language models are prone to "hallucinations" (generating content inconsistent with documents). Although Retrieval-Augmented Generation (RAG) technology can alleviate this problem, simple RAG is difficult to handle complex multi-step queries, so the value of multi-agent collaborative architecture is gradually emerging.

function-agent-rag uses a hybrid retrieval strategy:

1. Dual Retriever Architecture: Combines BM25 (based on term frequency statistics, good at precise keyword matching) and vector retrieval (based on semantic encoding, understands deep meanings) to balance precise matching and semantic understanding, improving retrieval recall and accuracy.
2. Property Graph Indexing: Extracts entities (names, organizations, concepts, etc.) and their relationships from documents, supporting structured information extraction, multi-hop reasoning (answering complex queries that require connecting multiple information points), and context enhancement.

The system divides the Q&A task into two roles:

• Researcher Agent: Receives user queries, uses dual retrievers and property graph indexing to find relevant document fragments, evaluates the relevance and reliability of information, and organizes it into a structured research report (can call tools such as keyword search, semantic search, graph query, etc.).
• Writer Agent: Generates the final answer based on the report provided by the researcher, focusing on organizing information into coherent and easy-to-understand content to avoid hallucination problems. The two agents collaborate through clear interfaces, similar to a human research team, improving the quality of Q&A and making their behavior transparent and interpretable.

function-agent-rag is suitable for multiple scenarios:

• Enterprise Knowledge Base Q&A: Helps employees quickly find internal documents (policies, technical documents, project reports, etc.).
• Academic Research Assistance: Assists researchers in understanding the core content of papers, finding relevant research or cross-domain connections.
• Legal Document Analysis: Quickly retrieves contracts, precedents, regulations, etc., to obtain precise legal basis.
• Medical Literature Query: Finds key information such as disease treatment plans and drug interactions from medical literature.

Compared with traditional RAG systems, function-agent-rag has the following advantages:

1. Improved Accuracy: The separated researcher-writer architecture ensures answers have reliable document basis.
2. Complex Query Handling: Property graph indexing supports multi-hop reasoning to handle complex problems that integrate multiple information sources.
3. Optimized Retrieval Quality: Dual retrievers balance the needs of precise matching and semantic understanding.
4. Scalability: The function call mechanism facilitates the expansion of new tools and data sources.
5. Transparent and Interpretable: Agents have clear division of labor, allowing users to trace the source of answers and the reasoning process.

function-agent-rag is an open-source project on GitHub, providing a reference implementation for RAG technology. Its modular design encourages community contributions, allowing developers to easily replace or extend components (such as embedding models, retrieval strategies, agent roles, etc.).

function-agent-rag represents the evolution direction of RAG technology from simple retrieval-generation mode to multi-agent collaborative mode. With the maturity of large language models and agent technology, such innovations will promote the development of document Q&A systems to a higher level.