VoucherVision: A Large Model-Driven Intelligent Transcription System for Natural History Specimen Labels

The VoucherVision project, launched by the University of Michigan Herbarium, uses large language model (LLM) technology to revolutionize the transcription process of natural history specimen labels. It addresses the problems of time-consuming and error-prone manual transcription, as well as the inability of traditional OCR to handle the diversity of labels, enabling automated data extraction and providing an efficient digitization solution for biodiversity research.

Natural history specimen labels record key information such as collection location and time, which are valuable resources for biodiversity research. However, traditional manual transcription is time-consuming, labor-intensive, and error-prone, leaving a huge number of specimens worldwide un-digitized. Moreover, labels come in various formats (handwritten/printed, multilingual, faded/damaged), making traditional OCR technology incompetent.

The project was initiated by the University of Michigan Herbarium (with over 1.7 million specimens). LLM was chosen for its multimodal understanding and structured extraction capabilities. The technical workflow includes: image preprocessing (denoising, enhancement, etc.) → LLM extraction of key fields (species name, collection information, etc.) → structured output → human review and feedback to optimize the model.

1. Multimodal understanding: Combine visual and semantic information to recognize the layout structure of labels;
2. Contextual reasoning: Complement ambiguous/missing information and mark low-confidence results;
3. Multilingual support: Handle multiple languages such as Latin and Chinese, and map corresponding scientific terms.

Accelerate biodiversity research (improve digitization efficiency, combine GIS to analyze species distribution and climate change impacts); Promote open data sharing (support Darwin Core standards and break data silos); Lower barriers (help small herbaria and institutions in developing countries with limited resources).

Handwriting recognition: Pretraining + fine-tuning + confidence evaluation; Label quality variation: Image preprocessing optimization (color correction, damage repair); Domain knowledge integration: Incorporate biological terms and nomenclature rules during training to ensure accuracy.

The project is open-source (code hosted on GitHub) for global institutions to use and improve; Establish a user community to share experiences, discuss issues, and contribute suggestions, driving the continuous evolution of the tool.

Future directions: Optimization of real-time processing, development of mobile applications, multimodal fusion (integrate specimen image information), and intelligent quality control to reduce human review.