video-to-text: An Intelligent Tool to Automatically Convert YouTube and Twitter/X Videos into Readable Articles

video-to-text is an open-source Python-based tool that can automatically transcribe and translate YouTube and Twitter/X videos into structured Brazilian Portuguese articles, and generate SEO-friendly static HTML pages. Its core features include local Whisper transcription and Claude/Gemma translation, addressing the pain point of low reading efficiency for video content, allowing users to read at their own pace and quickly locate key information.

In the era of information explosion, video content is growing rapidly, but watching long videos is time-consuming and does not allow quick browsing. The project was born because the developer prefers reading over watching long videos. The goal is to build an end-to-end pipeline: obtain content from video URLs, automatically transcribe, translate, and reorganize into well-structured articles, and present them as static HTML for easy mobile reading.

Adopts a modular architecture: Input layer receives URLs → Provider layer detects sources and invokes strategies → Processing layer uses Claude for translation and reorganization → Generation layer builds static HTML → Output layer presents. The Provider abstraction layer supports expansion; current strategies include YouTube (using youtube-transcript-api) and Twitter/X (download audio with yt-dlp + local transcription with mlx-whisper). After transcription, Claude processes: translate to Brazilian Portuguese, remove redundancy, filter ads, and split into chapters by topic.

Mobile-first design; static HTML without frameworks loads quickly. Supports three themes (Sépia default, bright, dark); progress tracking with automatic recovery, saved independently across devices; clickable chapter index for quick jumps; responsive layout adapts to small screens.

Integration with Hermes Agent: Users send links to Hermes → automatic processing → generate HTML and push → users receive links to read. Local deployment: Clone the repository, create a virtual environment, install dependencies, start the server; use the pipeline.py command to process videos, which automatically detects the URL source without needing to specify it.

Cases include popular AI field articles such as "Claude Code Lead Talks About the Future of Programming". Significance: Convert passive viewing to active reading, improve efficiency (reading is 2-3 times faster than watching videos), enable flexible fragmented reading, easy retrieval and archiving, and accessibility; demonstrate the idea of open-source tool combination and extension design.

The project accurately addresses the pain points of video reading, with elegant technical implementation (modular, pure static output), excellent user experience, and convenient integration. In the future, as the capabilities of large models improve, the application scenarios of such automated content conversion tools will become more extensive.