pdf2md: A PDF-to-Markdown Tool Based on Local Vision-Language Models

pdf2md is a single binary tool written in Go developed by cheto5144. Its core feature is using local Vision-Language Models (VLM) for PDF-to-Markdown inference without relying on external cloud services, making it suitable for privacy protection and offline use scenarios. This article will detail the tool from aspects such as background, solution, architecture, and applications.

Original Author/Maintainer: cheto5144 Source Platform: GitHub Original Link: https://github.com/cheto5144/pdf2md Source Publication/Update Time: 2026-05-31

PDF document format conversion is a classic challenge. Traditional tools face the following issues:

• Layout Loss: Complex layouts like tables, images, and columns become disorganized after conversion
• Format Recognition Difficulty: Semantic information such as heading levels, lists, and code blocks are hard to extract
• Dependency on External Services: Requires calling cloud APIs, leading to privacy and cost concerns
• Complex Installation: Needs many dependencies and tedious configuration

These problems are more prominent when converting to structured Markdown, such as in scenarios like building knowledge bases or preparing training data.

pdf2md uses local Vision-Language Models (VLM) to understand the visual layout of PDF pages and convert them into structured Markdown. Core design features:

1. Single Binary File: Standalone Go binary with no need for complex dependency installation
2. Local VLM Inference: All processing is done locally without relying on external cloud services
3. Visual Understanding Capability: Recognizes page layouts, table structures, image positions, etc.
4. Markdown Output: Preserves the original document's hierarchical structure and format information

The technical architecture is divided into three layers:

• PDF Rendering Layer: Converts PDF pages into image formats (rasterized bitmaps)
• VLM Inference Layer: Receives images and extracts structured information such as text content, heading levels, table structures, list/code block recognition, and image positioning
• Markdown Generation Layer: Converts VLM output into Markdown syntax, handling complex layouts like cross-page tables, nested lists, and mixed text-image arrangements

Applicable scenarios include:

• Academic Research: Converting papers to Markdown facilitates note-taking and citation management
• Enterprise Document Processing: Local batch conversion of sensitive reports/manuals to avoid data leakage
• Knowledge Base Construction: PDF-to-Markdown for search, indexing, and knowledge graph building
• AI Training Data Preparation: Generating high-quality structured training data while preserving the original document structure

Privacy Advantages:

• Data stays local, no cross-border transfer
• No network dependency, usable offline
• Controllable cost, no API call fees
• Compliance-friendly (adheres to GDPR, HIPAA, etc.)

Limitations:

• High computational resource requirements (VRAM and computing power)
• Conversion quality varies by document type (good for structured docs, low accuracy for complex/scanned docs)
• Markdown cannot perfectly preserve all PDF formats
• Conversion quality depends on the VLM model version

Comparison with Existing Solutions:

Feature	pdf2md	Traditional OCR Tools	Cloud PDF Conversion Services
Privacy Protection	Excellent (fully local)	Excellent (fully local)	Poor (requires document upload)
Installation Complexity	Low (single binary)	Medium to High	None (web service)
Layout Understanding	Strong (VLM visual understanding)	Weak (text extraction only)	Medium to Strong
Offline Availability	Yes	Yes	No
Cost	One-time (model download)	One-time	Pay-as-you-go
Processing Speed	Depends on hardware	Fast	Depends on network and service

Future Development Directions:

• Support more VLM backends
• Batch processing optimization (parallelism, progress monitoring)
• Interactive configuration (adjust conversion parameters)
• Expand output formats (HTML, JSON, etc.)
• Quality assessment tool (identify pages needing manual review)

pdf2md represents a new paradigm for solving complex layout problems using multi-modal large model visual understanding. Although not a perfect solution, its local-first, privacy-friendly design and clever application of VLM technology make it a powerful tool in specific scenarios. As VLM capabilities improve and computing costs decrease, such tools are expected to become mainstream in the future.