Integration of Biomedical Knowledge Graphs and Large Language Models: Technical Exploration and Practice of OntoLLM

This article explores OntoLLM, a technical approach for deep integration of Ontology and Large Language Models (LLMs). It aims to address the problems of insufficient knowledge accuracy and limited reasoning capabilities of LLMs in the biomedical field, while also overcoming the limitations of ontology in flexibility and scalability. The core idea is to leverage knowledge-enhanced pre-training strategies and hybrid reasoning architectures to achieve complementary advantages between structured knowledge and neural networks, thereby enhancing biomedical knowledge representation and reasoning capabilities. This approach has practical value in scenarios such as literature mining, clinical decision support, and drug development.

Advantages and Limitations of Ontology

Ontology is a formal knowledge representation method that is well-established in the biomedical field (e.g., GO, DO ontology libraries). It provides a standardized terminology system and hierarchical structure, supporting data source interoperability. However, it has limitations such as high construction and maintenance costs, reasoning relying on preset rules, and difficulty in processing unstructured text.

Potential and Challenges of LLMs

LLMs acquire rich linguistic and world knowledge through pre-training, enabling them to connect unstructured literature with structured knowledge. However, they are prone to generating 'hallucinations' (false information) and have a black-box decision-making process, which does not meet the requirements for accuracy and interpretability in the biomedical field.

bio-ontollm project was born to address the above dual dilemmas.

Knowledge-Enhanced Pre-Training Strategies

1. Ontology-Guided Masked Language Modeling: During pre-training, the model predicts both masked words and related ontology concepts, forcing it to learn language patterns and domain knowledge structures.
2. Aligned Learning of Concept Embeddings: Align ontology concept embeddings with the LLM's word vector space to improve the accuracy of term disambiguation.

Hybrid Reasoning Architecture

It combines symbolic reasoning and neural network reasoning: First, the LLM understands natural language queries and extracts key entities and relationships; then, it maps them to the ontology knowledge graph for rule-based reasoning; finally, it generates standardized answers based on the feedback results. This approach retains the flexibility of LLMs while ensuring knowledge accuracy and interpretability.

Biomedical Literature Mining

Using ontology knowledge to enable zero-shot/few-shot learning, identify new concepts not present in training data (e.g., inferring the association between new symptoms of rare diseases and known diseases), and assist researchers in discovering diagnostic and treatment clues.

Clinical Decision Support

Generate evidence-based clinical recommendations and provide reasoning chains to help doctors understand the basis of the plan; link electronic medical records with medical ontologies to identify medication conflicts, allergy risks, and personalized treatment opportunities.

Accelerated Drug Development

Integrate multi-source heterogeneous information (literature, patents, clinical trials) to build a drug-target-disease association network; predict compound side effects, drug interactions, and the possibility of repurposing old drugs to support drug repositioning.

Existing Challenges

1. Dynamic Knowledge Update: Need to timely absorb new biomedical knowledge while maintaining the stability of existing knowledge.
2. Cross-Ontology Knowledge Fusion: There are multiple heterogeneous ontology libraries in the biomedical field (e.g., GO, DO, SNOMED CT), requiring effective mapping and fusion mechanisms.
3. Interpretability and Credibility: Practical medical scenarios have strict requirements for the interpretability of the model's reasoning process and the credibility of results.

Future Directions

Explore incremental/continuous learning to achieve dynamic knowledge updates; introduce ontology alignment and knowledge graph fusion technologies to build a unified knowledge base; develop visualization tools and uncertainty quantification methods to enhance interpretability and credibility.

The bio-ontollm project represents an important exploration direction in the intersection of artificial intelligence and biomedicine. It emphasizes that while pursuing model scale and performance, attention should be paid to the structuring and interpretability of knowledge. The integration of ontology and LLMs is a feasible path to reliable and trustworthy medical AI.

It is recommended that practitioners engaged in biomedical informatics, knowledge graph construction, and medical AI application development deeply study and draw on the technical concepts and practical experience of OntoLLM.