Fusion of Knowledge Graph Embedding and Large Language Models: A Hybrid Reasoning Framework to Reduce LLM Hallucinations

This article introduces an end-to-end hybrid framework project that deeply integrates Knowledge Graph Embedding (KGE) with Large Language Models (LLMs) to reduce LLM hallucination issues by injecting structured knowledge. The project adopts a six-stage pipeline architecture, applied to a Spanish technical event management system, enabling advanced operations and reasoning on concept graphs. The core goal is to improve the accuracy and reliability of LLM responses, providing a reference architecture for knowledge-enhanced generative AI.

Large Language Models (LLMs) have made revolutionary progress in the field of NLP, but the hallucination problem (generating content that seems reasonable but is inconsistent with facts) limits their application in critical tasks. Traditional mitigation methods such as Retrieval-Augmented Generation (RAG) and prompt engineering rely on unstructured text, making it difficult to ensure knowledge accuracy and consistency. As a structured knowledge representation, knowledge graphs provide a verifiable and inferable knowledge base, offering new ideas for solving the hallucination problem.

The project's six-stage pipeline architecture is as follows:

1. RDF Parsing: Convert RDF graphs (about 60,000 records) into TSV triples (training/validation/test sets);
2. KGE Training: Use the PyKEEN library to train the TransE model (default, supports DistMult/ComplEx) with hyperparameters including 256-dimensional embedding, 600 training epochs, batch size of 2048, and 50:1 negative sampling ratio;
3. Link Prediction: Infer potential relationships between entities and output Top-K implicit relationship predictions;
4. Intelligent Event Creation: Combine Case-Based Reasoning (CBR), KGE, and conversational LLMs, supporting LLM-free mode (digital menu) and conversational mode (local LLM interaction);
5. Evaluation: Adopt multi-dimensional metrics such as Hit@k, CBR agent presence rate, recommendation completeness, Exact Match (EM), Token F1, and BERTScore.

Tech Stack

• Python 3.11
• PyKEEN (KGE library)
• vLLM (LLM inference service)
• Hugging Face (model hosting)
• Meta-Llama-3-8B-Instruct (default LLM)

Deployment Architecture

1. vLLM service: Run in an independent terminal with the command vllm serve meta-llama/Meta-Llama-3-8B-Instruct --port 8000 --dtype float16 --max-model-len 4096
2. Main application: Execute KGE training, link prediction, and event creation

Project Structure

• src/: Core code (configuration, implementation of each stage, evaluation module)
• data/: RDF graphs, triples, evaluation corpus
• out/: Model outputs, embeddings, prediction results, evaluation reports
• figuras/: Configuration guides and visualization resources

Corpus Generation

Generate about 3700 single-hop questions and 490 multi-hop chain questions via python src/generate_corpus.py, covering single-hop fact-based QA, multi-hop reasoning, and triple verbalization.

1. Hallucination Mitigation: Constrain the LLM generation space through structured knowledge injection; compared to text-only RAG, knowledge graphs provide more precise and verifiable knowledge sources;
2. KGE Application Expansion: Extend traditional KGE (link prediction/knowledge completion) to the fields of dialogue systems and content generation;
3. Multilingual Support: Choose Spanish as the working language, filling the gap in KGE-LLM fusion research for languages other than English.

1. Computational Resource Requirements: KGE training and LLM services require GPU support, leading to high deployment costs;
2. Domain Specificity: Currently optimized for the event management domain, migration to other domains requires adaptation;
3. Knowledge Graph Construction: Acquisition and maintenance of high-quality RDF graphs remain bottlenecks;
4. Latency Issue: The pipeline of KGE retrieval + LLM generation may introduce response latency.

Application Prospects

• Enterprise Knowledge Management: Integrate scattered knowledge into a unified graph to improve the accuracy of internal QA systems;
• Professional Domain Assistants: Ensure LLM suggestions comply with norms and facts in fields such as healthcare, law, and finance;
• Multilingual Knowledge Systems: Based on Spanish implementation, expand to global multilingual services.

Conclusion

The project successfully implements an end-to-end system for deep fusion of KGE and LLMs, effectively reducing LLM hallucinations and providing verifiable and interpretable reasoning capabilities. This technical path provides a reference for building reliable professional AI systems, indicating that the fusion of structured knowledge and generative AI is an important future direction.