Query-Tube-AI: A YouTube Video Semantic Search System Based on Transformer Embeddings

Query-Tube-AI is an open-source project that uses Transformer models to generate video embedding vectors for semantic retrieval of YouTube content, supporting multi-dimensional similarity ranking based on metadata and subtitles. It aims to address the pain point where traditional keyword search fails to meet users' needs for deep understanding of video content, allowing users to accurately locate desired video content using natural language descriptions.

In the era of information explosion, YouTube has become one of the world's largest video knowledge bases. However, traditional keyword search often fails to meet users' needs for deep understanding of video content. Users may remember a concept or viewpoint in a video but cannot accurately recall the keywords in the title or description. The Query-Tube-AI project was born to solve this pain point. By introducing the semantic understanding capability of Transformer models into the field of video retrieval, it allows users to accurately locate desired video content using natural language descriptions.

Query-Tube-AI adopts a modular code organization approach, with the project structure clearly divided into a data layer, a notebook experiment layer, and a script execution layer. The data directory is responsible for storing and managing raw metadata and subtitle text obtained from YouTube; the notebook layer provides an interactive environment for exploration and prototype verification; and the script layer encapsulates reusable data processing pipelines. This layered design ensures both R&D flexibility and convenience for production deployment.

In terms of technology selection, the project relies deeply on the Transformer ecosystem. Through pre-trained language models, the system can encode video titles, descriptions, and subtitle content into high-dimensional semantic vectors. These vectors capture the deep semantic features of text, making semantically similar but differently phrased content close to each other in the vector space. This embedding representation lays a solid foundation for subsequent similarity calculations.

The core innovation of the project lies in the unified mapping of multi-modal information of videos into a shared semantic space. For each video, the system not only extracts text features from titles and descriptions but also incorporates automatically generated subtitles into the analysis. As a complete textual transcription of video content, subtitles contain the richest semantic information. Through chunk processing and vectorization encoding, subtitles of long videos are converted into a series of semantic vectors, which not only retain the integrity of local context but also support fine-grained content retrieval.

The embedding generation process fully considers the content characteristics of the YouTube platform. Video metadata often includes carefully written titles and descriptions by creators, which highly summarize the video's theme; while subtitles provide word-by-word content details. The system organically combines the two to construct a multi-level semantic representation of videos, supporting both theme-based coarse-grained retrieval and content-based precise positioning.

In the retrieval phase, Query-Tube-AI uses cosine similarity as the core metric. After the user's query text is encoded by the same embedding model, it is compared in batches with the semantic vectors of all content in the video library. The similarity score reflects the relevance between the query intent and the video content, and the system returns the most matching result list based on this.

To improve retrieval efficiency, the project may use vector indexing technology to accelerate query responses for large-scale datasets. When the video library scale reaches tens of thousands or even hundreds of thousands, the linear complexity of brute-force search will be difficult to meet real-time requirements. Through approximate nearest neighbor search algorithms, the system can reduce the query complexity from linear to logarithmic or even constant level while maintaining a high recall rate.

Query-Tube-AI has a wide range of application scenarios. For learners in the education field, it can help quickly locate teaching videos containing specific knowledge points; for content creators, it can discover high-quality reference materials related to their field; for researchers, it provides an efficient tool for video material research. Compared with YouTube's native keyword search, semantic search can understand the deep intent of the query, and even if the query words do not appear in the video description, it can return truly relevant results.

The project also demonstrates how to apply modern NLP technology to practical vertical domain problems. By combining the general semantic understanding capability of pre-trained models with domain-specific data, developers can build powerful dedicated search systems without training large models from scratch. This 'pre-training + fine-tuning/application' paradigm has become the mainstream path for current AI application development.

Query-Tube-AI represents an important development direction of video content retrieval technology. It proves the feasibility and effectiveness of combining cutting-edge NLP technology with traditional information retrieval. For developers who want to build vertical domain video search engines, this project provides a valuable reference implementation. With the continuous progress of multi-modal large model technology, future video search systems will be able to understand visual images, audio content, and text information simultaneously. The semantic retrieval foundation laid by Query-Tube-AI will become an important cornerstone of this evolution.