ai-arxiv-daily: A Practical Tool for Automatically Tracking Cutting-Edge AI Papers

This post introduces the ai-arxiv-daily open-source project, which automatically tracks the latest AI/LLM papers on arXiv daily, covering popular directions like large language models, AI agents, and RAG. It addresses the pain point of time-consuming manual paper browsing and filtering, helping users efficiently stay updated on field developments.

ai-arxiv-daily is an open-source automated paper tracking system whose core objective is to help users efficiently obtain the latest research results in the AI/LLM field. It was born out of the challenges faced by AI researchers: the huge number of daily papers on arXiv makes manual filtering inefficient and prone to missing important work. The system automatically crawls new arXiv papers via scheduled tasks, classifies and filters them based on preset keywords, and simplifies the process into daily pushes to improve efficiency.

Covers multiple popular directions in the current AI field:

• Large Language Models: Tracks mainstream model improvements, new architectures, scale expansion, etc.;
• AI Agents: Focuses on agent systems with autonomous planning and tool usage capabilities;
• RAG: Covers vector retrieval, knowledge base construction, etc., to solve large model hallucination issues;
• Prompt Engineering: Includes prompt design, in-context learning, chain-of-thought, etc.;
• RLHF: Covers alignment technologies like reward model training and preference learning;
• Multimodality: Research on integrating text, images, and other modalities;
• Code Generation: Program synthesis, code completion, etc.;
• Model Fine-tuning: Domain adaptation technologies like parameter-efficient fine-tuning and instruction fine-tuning.

The technical architecture is concise yet fully functional. It uses the arXiv API to query new papers regularly, and generates structured reports through keyword matching and NLP-based scoring. Workflow steps:

1. Daily query of new papers in relevant arXiv categories (cs.AI, cs.CL, etc.);
2. Extract metadata such as title, abstract, authors;
3. Filter relevant papers using keyword matching and text similarity algorithms;
4. Organize and output in formats like Markdown or email.

Value for different user groups:

• Academic Researchers: Quickly understand peers' work, avoid duplicate research, and find collaboration opportunities;
• Industrial Developers: Obtain practical technical solutions and reference the latest model architectures;
• Tech Enthusiasts: Systematically learn AI knowledge and build domain awareness;
• Research Teams: Deploy internal versions, customize directions, and integrate collaboration workflows.

The current version relies on keyword matching, and semantic understanding and personalized recommendations need improvement; in the future, text embedding and recommendation algorithms can be introduced to provide intelligent recommendations based on reading history. Additionally, there is a lack of paper quality evaluation (impact, innovation), which can be addressed by integrating citation data, author influence, conference rankings, etc., for comprehensive sorting.

ai-arxiv-daily uses automation to solve the problem of information overload, allowing researchers to focus on creative work. As AI develops, the importance of such tools becomes increasingly prominent. It is recommended that readers try using it or draw inspiration to build their own tracking systems—efficient information acquisition is a core competitive advantage.