Hybrid Framework of KGE and LLM: Reducing Large Model Hallucinations with Knowledge Graph Embeddings

This article presents an end-to-end system combining Knowledge Graph Embeddings (KGE) and Large Language Models (LLM), with the core goal of constraining LLM generation by injecting structured knowledge to reduce hallucination issues. The system has been validated in the scenario of Spanish technical incident management, featuring a six-stage fusion architecture. Evaluations show significant improvements in factual accuracy, multi-hop reasoning ability, and interpretability.

LLMs tend to produce 'hallucinations' (content that seems plausible but is factually incorrect) when generating text, which is extremely harmful in scenarios requiring precise knowledge such as healthcare and law. Traditional solutions like Retrieval-Augmented Generation (RAG) and fine-tuning rely on unstructured text corpora, making it difficult to ensure knowledge accuracy and consistency. As a structured knowledge representation, knowledge graphs can provide clear factual relationships and verifiable sources.

The system is built around the Spanish technical incident management scenario, processing a knowledge graph with approximately 60,000 incident records. The core process consists of six stages: 1. Parse RDF graphs into triples and split into datasets; 2. Train a DistMult KGE model using PyKEEN; 3. Perform link prediction with the KGE model to obtain implicit relationships; 4. Deploy the LLM (Meta-Llama-3-8B-Instruct) locally and inject knowledge context; 5. Dynamically configure session subgraphs based on case-based reasoning; 6. Conduct comprehensive verification and evaluation.

Knowledge Graph Structure: Includes entities such as incidents, technicians, customers, and explicit relationships; Embedding Learning: DistMult model (256-dimensional embeddings, 200 training epochs, batch size 2048, 100x negative sampling) trained on A100 GPU; LLM Service: Local service deployed with vLLM, providing OpenAI-compatible API; Interactive Session: Supports dynamic switching of incident contexts and automatic update of subgraph knowledge injection.

The system was evaluated in the Spanish technical incident management scenario, with a corpus containing 3700 single-hop questions and 490 multi-hop reasoning chains. Results show: 1. Improved factual accuracy, more accurate entity recognition and relationship inference, and significantly reduced hallucinations; 2. Enhanced multi-hop reasoning ability, where KGE link prediction supplements implicit relationships to support complete reasoning chains; 3. Improved interpretability, as KGE provides clear and traceable knowledge sources.

This framework provides a systematic solution for reducing LLM hallucinations in knowledge-intensive applications, with three key advantages over prompt engineering or RAG: 1. Precise injection of structured knowledge, constraining LLM generation within verifiable knowledge boundaries; 2. KGE mines implicit relationships to expand knowledge boundaries; 3. Customizable process, adaptable to different fields such as healthcare and law.

Current Limitations: KGE training requires large computing resources such as A100 GPU; knowledge graph construction and maintenance are complex; performance depends on the quality and coverage of the original graph. Future Directions: Explore efficient KGE training (e.g., knowledge distillation); automate graph updates; expand to multilingual scenarios; integrate advanced LLM capabilities such as tool use and multimodality.