KothaSet: An Open-Source CLI Tool for Generating High-Quality Training Datasets Using LLMs

In the era of large models, data quality often determines the final outcome more than model architecture. Whether it's supervised fine-tuning (SFT) or preference alignment (DPO/RLHF), high-quality training data is indispensable, but manual annotation is time-consuming and costly. KothaSet is an open-source command-line tool developed in Go that uses LLMs as teacher models to generate high-quality datasets. It supports multiple data formats and providers, making it suitable for model fine-tuning scenarios.

In large model training, data quality is a key factor. Scenarios like SFT, DPO/RLHF all require high-quality data, but manual annotation is time-consuming and costly, which has become a bottleneck restricting model training efficiency.

KothaSet is a CLI tool developed in Go, designed to simplify the generation of LLM training datasets. Its core features include:

1. Multi-provider support: Natively compatible with OpenAI API and local deployment solutions like DeepSeek, vLLM, and Ollama;
2. Flexible data formats: Supports four schemas: Instruction (Alpaca style), Chat (ShareGPT style), Preference (for DPO/RLHF), and Classification;
3. Engineering features: Streaming output, resume from breakpoints, JSONL format output, and fixed random seeds to ensure reproducibility.

KothaSet uses a dual-file configuration system:

• kothaset.yaml: Public configuration, including shared settings, context, and generation instructions, which can be committed to Git repositories;
• .secrets.yaml: Private configuration, storing sensitive information like API keys, with permissions set to read/write only for the owner (0600). This separated design balances security and team collaboration needs.

KothaSet is suitable for multiple scenarios:

1. Domain data expansion: Generate synthetic data for scarce domains;
2. Instruction data construction: Quickly generate diverse instruction-response pairs;
3. Preference data collection: Generate quality-distinguished preference pairs for DPO training;
4. Multilingual data: Generate cross-language training sets via configuring multilingual prompts.

The project is developed in Go, which brings advantages like simple deployment of compiled single binaries, high execution efficiency, and low resource consumption. In addition, built-in rate limiting avoids API throttling, and details like automatic parent directory creation and input file support enhance user experience.

KothaSet provides a professional and reliable solution for LLM data generation. Compared to simple scripts, it is more comprehensive in terms of data format standardization, resume from breakpoints, configuration management, etc., making it suitable for teams that generate training data at scale. In the future, adding automatic data quality verification and multimodal data support will further expand its application scope.