Sparrow: An Open-Source Platform for Enterprise Document Intelligence and Multi-Agent Workflows

Sparrow is an API-first open-source document intelligence platform for enterprise scenarios. Its core features include: support for local deployment (data privacy first), integration of Vision LLM and multi-agent workflows, and structured data extraction from complex documents such as invoices, reports, and tables. The project is maintained by Katana ML, open-sourced on GitHub (link: https://github.com/katanaml/sparrow), follows the GPL-3.0 license, and is suitable for industries with high data privacy requirements like finance and healthcare.

Background: Traditional OCR tools have single functions, and cloud document APIs have data export risks, making it difficult to meet the privacy compliance requirements of industries like finance and healthcare. Core design: Localization first (all inference done on own infrastructure), modular architecture (flexible combination of Vision LLM, Text LLM, and Agent capabilities), covering the entire process from simple extraction to complex decision-making.

Core capabilities: 1. Structured data extraction API (define Schema to automatically extract JSON); 2. Instruction processing (natural language verification such as invoice amount consistency); 3. Multi-agent workflow orchestration (supports complex business processes like accounts payable processing); 4. Multi-backend support (MLX, vLLM, Ollama, etc., adapted to different hardware). Technical components: Sparrow ML LLM (API engine), Sparrow Parse (Vision LLM library), Sparrow Agents (workflow), Sparrow OCR (text recognition), Sparrow UI (visual interface).

1. Bank statement processing: extract bank information, transaction details, balance summary, etc.; 2. Financial report table extraction: handle cross-row/merged cells and output standard JSON arrays; 3. Invoice processing: extract fields like invoice number and amount, support intelligent cropping to improve accuracy.

Deployment steps: Clone the repository → Install dependencies → Start the API (macOS requires additional installation of poppler). Command line example: Use sparrow.sh to submit documents, specifying parameters like Schema, pipeline, and model. Supported models include Qwen2.5 VL, Mistral, etc. Web UI features: Drag-and-drop upload, real-time result viewing, JSON Schema definition, and visual annotation.

Enterprise features: Rate limiting, usage analysis, commercial licensing. Competitor comparison: Compared to traditional OCR (e.g., Tesseract) or cloud APIs (AWS Textract), Sparrow's advantages are full localization, replaceable models, programmable workflows, and controllable costs; compared to models like LayoutLM, it provides higher-level abstraction and can be used in production without fine-tuning.

1. Hardware requirements: Vision LLM requires large GPU memory (e.g., Qwen2.5-VL-72B-4bit model); 2. Document adaptability: Recognition rate may decrease for extremely complex/non-standard formats (handwritten, severely distorted); 3. Learning curve: Agent workflow configuration requires understanding concepts like state management and error retry.

Sparrow is an important evolution direction in the document intelligence field, balancing ease of use and flexibility, with a localization strategy that meets privacy needs. Recommendations: Technical teams should prioritize it when evaluating document automation solutions; it can be used as a prototype or production infrastructure.