OCRPolish: A Toolkit for LLM-Optimized OCR Post-Processing and Metadata Extraction

OCRPolish is an OCR post-processing toolkit written in Python, optimized for OCR outputs processed by LLMs. Its core features include cleaning OCR text with messy formatting, extracting metadata via local LLMs, generating Obsidian index pages, etc. Its design goal is to upgrade raw OCR outputs into structured knowledge bases, making it particularly suitable for Obsidian users, researchers, and archival digitization scenarios, bridging the gap between OCR outputs and usable knowledge.

Traditional OCR outputs have issues such as messy formatting (residual headers/footers, line break errors), broken paragraphs, missing metadata, and difficulty in entity recognition. Although users often use LLMs to process OCR text after their popularization, general LLM prompt engineering cannot fully leverage the structured features of documents. Therefore, OCRPolish is designed for this scenario to address the above pain points.

OCRPolish's core features include:

1. Three-Layer Tag System: Concept layer (core topics), entity layer (states/regions/organizations, etc.), theme layer (NATO taxonomy hierarchical tags), supporting multi-dimensional indexing.
2. Obsidian Export: Generate Markdown files with YAML frontmatter (title, summary, date, etc.) and summary highlight blocks, adapting to Obsidian graph view and backlinks.
3. Three Commands: clean (text cleaning), metadata (metadata extraction), index (index generation).

Metadata extraction relies on local LLMs (default: Gemma4:31b via Ollama) with the following steps:

1. Load predefined NATO theme taxonomy (--hierarchy-file);
2. Filter via tag whitelist (--tags-file);
3. Automatically recognize entities and label them hierarchically;
4. Extract flattened standardized concept keywords. The results are both machine-readable (YAML) and human-readable (Obsidian tags/links).

Applicable Scenarios:

• Academic research: Process scanned papers to build searchable knowledge bases;
• Archival digitization: Convert historical documents into structured archives;
• Intelligence analysis: Document theme classification and entity extraction;
• Obsidian users: Seamlessly integrate OCR outputs into workflows. Development Quality: Use ruff, mypy, and pytest to ensure code quality and type safety.

Features:

• Obsidian-optimized output structure;
• Local LLM-driven, no API key required;
• Three-layer tag system supporting multi-dimensional indexing;
• Complete PDF mirror and link generation. Limitations:
• Depends on Ollama local runtime, requiring sufficient hardware resources;
• Default Gemma4:31b has VRAM requirements;
• Theme hierarchy and tags need to be pre-defined by users.

OCRPolish is not a general OCR tool, but a specialized optimization tool for the OCR→LLM→knowledge base workflow. It provides Obsidian users and researchers with a zero-API-cost, fully private document processing solution, converting raw OCR outputs into structured, searchable, and linkable knowledge bases, bridging the gap between OCR outputs and usable knowledge.