Fusion of LLM and Knowledge Graph: Building an Interpretable Structured Information Retrieval System

This article introduces an open-source project that deeply integrates large language models (LLMs) with knowledge graphs. Using RAG architecture and graph reasoning techniques, it aims to reduce LLM hallucination issues, improve the accuracy and interpretability of structured information retrieval, and provide practical references for building trustworthy AI question-answering systems. The core idea of the project is to use the semantic understanding ability of LLMs to extract structured knowledge, combine with the explicit relationship representation of knowledge graphs for reasoning, and generate accurate and traceable answers.

LLM Hallucination Challenges

Large language models perform strongly in natural language understanding and generation, but in precise scenarios such as medical care and law, the hallucination problem (generating incorrect but confident information) is unacceptable.

Limitations of Traditional RAG

Standard RAG enhances factuality through external knowledge bases, but has limitations: retrieved texts lack structured entity relationships, complex multi-hop reasoning has poor logical consistency, and answers have low interpretability (users cannot trace the source of conclusions).

The project proposes a deeply integrated architecture of LLM and knowledge graph, with the process divided into six stages:

1. Document Loading and Intelligent Chunking: Supports PDF/TXT input, uses RecursiveCharacterTextSplitter for chunking (500 tokens per chunk + 50 token overlap) to balance information density and context.
2. LLM-Driven Triple Extraction: Mistral-7B (deployed locally via Ollama) extracts head entity, relationship, and tail entity triples in JSON format from text chunks, converting unstructured text into machine-understandable graph structures.
3. Context Proximity Enhancement: Supplements statistical co-occurrence analysis; concepts co-occurring in the same text chunk are treated as implicit associations, adding a statistical dimension to edge weights.
4. Edge Merging and Graph Construction: Merges semantic edges and co-occurrence edges, aggregates weights of duplicate relationships, and uses NetworkX to build the graph (nodes = entities, edges = relationships, weights = relationship strength).
5. Community Detection and Visualization: Girvan-Newman algorithm detects concept clusters; PyVis generates interactive HTML visualizations (supports zooming, dragging, and detail viewing).
6. CSV Caching and Scalability: Persists relationships and chunking results as CSV, supporting resumable transfer and tool integration.

Key technology choices for the project:

• Embedding-free model: Relies on the structured extraction capability of LLMs, avoiding semantic drift of vector embeddings and simplifying the architecture.
• Local LLM deployment: Mistral-7B runs locally via Ollama, ensuring data privacy and controllable latency.
• LangChain framework: Uses abstractions such as document loading, text splitting, and chain calls to reduce development complexity.
• Pure Python stack: NetworkX (graph computing), PyVis (visualization), Pandas/NumPy (data processing), lightweight and easy to deploy.

The core value of the system is to convert unstructured documents into interactive knowledge graphs. Application scenarios include:

1. Medical literature analysis: Extracts disease-symptom-drug-side effect relationships to assist clinical decision-making.
2. Enterprise knowledge management: Explicitly represents implicit knowledge in PDFs/technical documents and builds organizational knowledge maps.
3. Research literature review: Automatically extracts key concepts and relationships to generate an overview of domain knowledge structures.
4. Interpretable question-answering: Graph-based question-answering can trace the source path of answers, providing stronger interpretability.

Existing Limitations

1. Extraction accuracy depends on LLM: The quality of triples is limited by Mistral-7B's capabilities; complex relationships or domain terms are prone to errors.
2. Lack of entity alignment: Cross-document entity disambiguation is not implemented; different expressions of the same entity are treated as different nodes.
3. Limited reasoning ability: Focuses on knowledge extraction and visualization, does not support complex reasoning such as multi-hop queries and path searches.
4. Scalability bottleneck: As the document scale grows, the LLM extraction stage easily becomes a performance bottleneck.

Improvement Directions

Corresponding to the limitations, we can optimize the LLM model, add an entity alignment module, expand reasoning functions, and implement parallelization and incremental processing.

Practical Insights

1. Structured first: Converting text to knowledge graphs increases complexity but improves accuracy and interpretability.
2. Value of hybrid methods: Combining LLM semantic understanding with the completeness of statistical methods to build a robust knowledge extraction process.
3. Importance of interpretability: In key scenarios, users need to understand the source of answers, and graphs provide a natural foundation.
4. Local deployment is feasible: Open-source models + local frameworks can balance privacy and AI capabilities.

Future Outlook

The LLM+KG fusion architecture will play a greater role in enterprise knowledge management, scientific research, medical decision-making, and other fields, and this project provides a practical open-source reference.