Few-Shot Learning Practice of Large Language Models in Biomedical Relation Extraction

This article introduces the open-source project few-shot-biore, which aims to explore the few-shot learning capabilities of open-source large language models for Biomedical Relation Extraction (BioRE) tasks and compare the effectiveness and feasibility of traditional supervised learning methods. The project provides a complete experimental framework and evaluation system, offering practical references for the field of biomedical natural language processing.

Biomedical Relation Extraction (BioRE) is a key technology for automatically identifying semantic relationships between entities from biomedical literature. Traditional methods rely on large amounts of labeled data for supervised learning, but the annotation cost in the biomedical field is extremely high and requires professional knowledge. Few-shot learning leverages the pre-trained knowledge of large language models and can extract specific relationship types with only a small number of examples, providing a new approach to solving the annotation bottleneck.

few-shot-biore is an open-source research project, accompanied by the paper Few-Shot Biomedical Relation Extraction with Large Language Models: A Viable Alternative to Supervised Learning?, which systematically compares the performance differences between prompt engineering and supervised learning. Its core features include: evaluation based on the BioREDirect standard dataset; support for multiple open-source large language models; a complete pipeline (from data parsing to result evaluation); and modular code for easy reproduction and expansion.

The project adopts a three-stage pipeline architecture:

1. Data preprocessing: Use parse.py to convert the PubTator format of the BioREDirect dataset into structured JSON;
2. Relation extraction: extract.py loads the large language model and performs extraction through carefully constructed few-shot prompt templates;
3. Evaluation: The evaluate directory provides standardized scripts to calculate metrics such as precision, recall, and F1 score.

1. Few-shot prompt design: Select representative examples from the training set, construct prompts containing input text, entity pairs, and relationship labels to guide the model to understand the semantic patterns of biomedical relationships without fine-tuning parameters;
2. Open-source model support: A model-agnostic architecture that can integrate multiple open-source large language models from the Hugging Face ecosystem, enabling flexible exploration of the relationship between model capabilities and task performance.

1. Reducing annotation costs: The few-shot method can achieve results similar to traditional supervised learning with only dozens of examples, significantly lowering the threshold for domain annotation;
2. Accelerating research iteration: Without training model parameters, it is possible to quickly try different prompt strategies, example selection methods, and model configurations;
3. Promoting domain transfer: The general semantic capabilities of large language models can be easily transferred to new relationship types or biomedical subfields.

Usage steps:

1. Install dependencies: pip install -r requirements.txt;
2. Download the dataset: wget https://ftp.ncbi.nlm.nih.gov/pub/lu/BioREDirect;
3. Run data parsing: python parse.py;
4. Perform relation extraction: python extract.py;
5. Evaluate results: Use the scripts in the evaluate directory.

few-shot-biore provides a practical open-source benchmark for the field of biomedical relation extraction, demonstrating the potential of open-source large language models in few-shot scenarios. With the improvement of model capabilities and data accumulation, few-shot learning is expected to become a viable alternative to traditional supervised learning (especially in scenarios with limited annotation resources). The project provides complete code implementation and an evaluation framework for domain researchers and developers, which is worth referencing and reusing.