ProjectScriber 2.0: A Powerful Tool for Intelligently Packaging Project Code for LLMs

ProjectScriber 2.0 is a command-line tool designed for LLM code processing scenarios, addressing core pain points in context delivery when using LLMs for code analysis, refactoring, or documentation generation. It filters relevant code modules through intelligent dependency analysis and whitelist mechanisms, supports Rust acceleration to improve performance, and provides token budget control functionality, ultimately generating a single structured context file suitable for LLM processing.

In LLM code tasks, directly passing the entire codebase leads to problems like token waste and noise introduction. ProjectScriber emerged as a solution—it is not a simple file merging tool, but automatically filters the most relevant modules based on entry files by understanding code dependencies, generating clean and complete context documents to enhance LLM output quality.

Core Design Philosophy

• Whitelist Priority: By default, only explicitly identified code files are included, automatically excluding irrelevant content such as binaries and lock files to ensure output purity.
• Intelligent Scoring Engine: Analyzes the project dependency graph (e.g., Python import relationships), calculates the relevance score of files relative to the entry point, and prioritizes retaining high-relevance files.

Technical Architecture Highlights

• Rust Native Acceleration: The underlying layer uses Rust to implement I/O and directory scanning, ensuring efficient processing of large codebases. Source code compilation is supported (requires Rust 1.70+).
• Dual-Language Development: Python handles high-level logic and user interfaces, while Rust processes performance-critical paths, balancing development efficiency and execution speed.
• Token Budget Control: Set an upper limit via the --max-tokens parameter, intelligently select relevant files, and control API costs.

Features

• Intelligent Project Mapping: Generates a tree structure to clearly display included/excluded files.
• Dependency Graph Analysis: Parses Python import statements to build module dependency relationships.
• Flexible Configuration: Customize matching rules, token estimation parameters, ignored directories, etc., via the [tool.scriber] block in pyproject.toml.
• Real-Time Feedback: Provides progress bars and statistical reports for an intuitive understanding of processing status.

Installation and Usage

• Installation: pip install project-scriber or uv pip install project-scriber
• Quick Workflow: Initialize configuration (scriber --init) → Package with entry point (scriber src/main.py --output context.md) → View statistics → Submit to LLM.

Use Cases

• Code Review/Refactoring: Specify module entry points to generate streamlined context for LLM analysis.
• New Project Onboarding: Generate structured project maps to quickly grasp the architecture.
• Documentation Generation: Create focused and accurate technical documents based on filtered context.

Comparison with Similar Tools

Compared to simple file merging tools (e.g., find+cat) or export scripts, ProjectScriber has the following advantages:

1. Intelligence: Understands code structure instead of just merging files;
2. Controllability: Precisely controls output through token budget and relevance thresholds;
3. Professionalism: Optimized for LLMs, generating standardized Markdown;
4. Performance: Rust acceleration adapts to large projects.

Limitations

• Language Support: Python dependency analysis is the most comprehensive; support for other languages is limited;
• Complex Structures: Multi-package repositories, symbolic links, etc., require additional configuration;
• Token Estimation: Character-to-token conversion is an estimated value; actual results depend on the model.

ProjectScriber is a typical example of intelligent developer tools. It not only enables code packaging but also serves as a complete solution for AI-assisted programming by understanding code and optimizing context. As LLMs become more integrated into development, such tools will grow in importance. Its MIT open-source license allows for community contributions and customization, and future versions are expected to support more languages and more intelligent analysis features.