miku-text-bundle: A Text Collection and Chunking Tool for Codebases Targeting Generative AI

miku-text-bundle is a Node.js CLI tool developed by Toshiki Iga (@igapyon), designed to address context window limitations and tedious operations when passing codebases to generative AI. This tool supports intelligent file collection, multi-encoding handling, smart chunking, .gitignore integration, and TODO extraction, helping developers efficiently feed code to AI for tasks like code review, refactoring, and documentation generation. The tool's source code is hosted on GitHub (https://github.com/igapyon/miku-text-bundle), and it was released on 2026-06-06.

With the widespread application of large language models like ChatGPT and Claude in software development, developers often need to provide project code as context to AI assistants. However, directly copying and pasting large amounts of files is both tedious and easily exceeds the model's context window limit, restricting the efficient use of AI in scenarios like code review and refactoring suggestions.

Intelligent File Collection

Automatically traverses directories, excluding non-text content such as version control (.git), IDE configurations (.idea/.vscode), build outputs (build/dist/target), as well as media, compressed, and binary files.

Multi-Encoding Support

Default is UTF-8; Shift_JIS is optional, and encoding can be specified for specific extensions.

Smart Chunking

Splits prioritizing file boundaries; if a single file exceeds the limit, it splits by lines and generates an index file. The chunk size can be set via the --max-chars parameter.

.gitignore Integration

Automatically reads .gitignore to exclude files, ensuring content consistency with version control.

TODO Extraction

Scans for TODO/FIXME/XXX markers in code and summarizes them into the index file.

Use Case 1: Code Review and Refactoring

Command: miku-text-bundle --input . --output ./ai-review to generate bundles for AI to review code quality.

Use Case 2: Generate Project Documentation

Command: miku-text-bundle --input ./src --output ./docs-bundle --max-chars 80000 for generating documentation materials.

Use Case 3: Legacy System Analysis

For Japanese legacy projects: miku-text-bundle --input ./legacy-app --output ./analysis --encoding shift_jis --add-exclude-directory "logs,backup".

Use Case 4: Interview Preparation

Package open-source projects: miku-text-bundle --input ./awesome-project --output ./study for AI to explain principles.

File Sorting Strategy

Converts paths to POSIX format and sorts them in ascending order of Unicode code points of UTF-16 encoding units to ensure output consistency.

File Content Format

Each file is presented with a level-3 heading (path) + code fence (specifying language), preserving the original format.

Statistical Information

Outputs concise statistics (number of collected files, skipped files, etc.) after execution; use --verbose to view detailed exclusion reasons.

Comparison with tar/zip

Advantages: Native Markdown can be directly pasted, smart chunking adapts to context windows, automatic syntax highlighting, clear directory structure, and TODO extraction.

Comparison with IDE's "Copy as AI Prompt"

Advantages: Batch processing of entire projects, offline availability, versionable generated bundles, and flexible configuration.

Best Practices

1. Adjust the --max-chars parameter according to the target AI model;
2. Optimize the exclusion list via --add-exclude-directory/extension;
3. Add the bundle directory to .gitignore;
4. Create shortcut scripts to simplify operations.

Limitations

1. Super large projects still require multiple interactions;
2. Cannot handle binary files;
3. Encoding needs to be explicitly specified;
4. Risk of sensitive information leakage (need to check configuration files).

miku-text-bundle addresses common pain points for developers using generative AI, making it more efficient to feed code to AI through intelligent collection and chunking. As AI-assisted programming becomes more popular, the importance of such tools will increasingly stand out, helping developers focus more on creative collaboration with AI and optimize their workflows.