inference-research: A Daily Intelligence System for Automated LLM Inference Optimization Research

inference-research is an automated research project developed by sara4dev, aiming to address the information overload issue in the field of LLM inference optimization. Inspired by Andrej Karpathy's autoresearch, the project runs Claude Code via daily scheduled tasks to track the latest papers, blogs, and code commits of mainstream inference frameworks like vLLM, SGLang, and TensorRT-LLM, and generates actionable research reports using Musk's Five-Step Method (first principles thinking).

Information Explosion in LLM Inference Optimization

With the rapid development of large language models, inference optimization has become a core battlefield in AI infrastructure. Projects like vLLM, SGLang, and TensorRT-LLM generate a large number of code commits, papers, and technical blogs daily. Manual tracking requires significant time, leading to a prominent information overload problem.

Rise of Automated Research

Andrej Karpathy's autoresearch demonstrated the possibility of AI-assisted research. sara4dev applied this concept to the field of inference optimization, which has stronger engineering practices, to create a dedicated automated intelligence system.

Scheduled Task-Driven

The core of the project is a daily scheduled task (cron job) triggered by run-daily.sh, which calls Claude Code to execute research tasks. The advantages of choosing Claude Code include: code understanding ability, multimodal analysis, structured output, and automated integration.

Target Project Coverage

The research scope focuses on five influential projects:

Project	Maintainer	Core Features
vLLM	Open-source community	High throughput, PagedAttention, extensive ecosystem
SGLang	LMSYS	Structured generation, RadixAttention, multimodal
TensorRT-LLM	NVIDIA	Production-grade optimization, GPU kernel optimization, quantization support
NVIDIA Dynamo	NVIDIA	Inference service framework, dynamic batching, multi-model support
LLM-D	Open-source community	Distributed inference, scheduling optimization, workload management
These projects represent different technical paths, from kernel optimization to service-layer scheduling, and from single-machine to distributed deployment.

Information Collection Phase

The daily workflow starts with information collection:

1. Code commit tracking: Monitor the latest commits in target repositories and analyze the significance of code changes
2. Paper retrieval: Search for inference optimization-related papers on arXiv and in conferences
3. Blog monitoring: Track official project blogs and releases from technical teams
4. Community dynamics: Follow GitHub issues and discussions

First Principles Analysis (Musk's Five-Step Method)

The collected information is deeply analyzed using the five-step method:

1. Question the requirement: For example, when an optimization solution claims to need a complex scheduling algorithm, question whether the requirement is reasonable
2. Remove components: Consider whether steps/components can be removed, such as whether complex batching can be eliminated through other means
3. Simplify and optimize: Optimize the efficiency of remaining parts on a streamlined architecture
4. Accelerate iteration: Focus on development iteration speed (build, test, deployment time)
5. Automate: Automate repetitive tasks, including the project's own information collection and analysis

Daily Reports

Results are saved in Markdown format in the reports/ directory (file names include dates, e.g., 2026-04-04.md), and the content includes: executive summary, project updates, in-depth analysis, action recommendations, and related resources.

Baseline Reports

The baseline/ directory contains initial in-depth research reports, which serve as a reference benchmark for subsequent work and provide comprehensive technical analysis of each framework.

Notifications and Version Control

After reports are generated, notifications are pushed via Telegram; daily reports are automatically committed to the Git repository, forming a traceable research history.

Researchers

• Information filtering: Sift important progress from massive information
• Trend identification: Identify technical trends through daily reports
• Source of inspiration: First principles analysis inspires new research directions
• Competitive intelligence: Understand the pros and cons and evolution of different technical paths

Engineers

• Best practice updates: Obtain the latest optimization techniques from various projects
• Problem solutions: Discover solutions to common problems from community discussions
• Technical selection reference: Make informed decisions based on comprehensive comparisons
• Performance optimization inspiration: Get optimization ideas from papers

Learners

• Structured knowledge: Understand the overall picture of the field through reports
• Latest progress: Keep up with cutting-edge technology
• Analytical methods: Learn to analyze technical problems using first principles thinking
• Resource index: Report links form a learning resource library

Current Limitations

• Language limitation: Mainly focuses on English resources, which may miss progress from other language communities
• Trade-off between depth and breadth: Daily reports prioritize timeliness, which may sacrifice depth
• Verification challenge: AI-generated analysis requires manual verification and may have understanding biases

Potential Improvements

• Multilingual support: Integrate translation capabilities to cover more language resources
• Interactive exploration: Add query functions to dive deep into specific topics
• Community contributions: Open report contribution mechanisms to gather community wisdom
• Visualization enhancement: Add trend charts and technical evolution visualizations