AggTradesTool: A Complete Toolchain for Preparing Binance Aggregated Trade Data for Machine Learning

AggTradesTool is an open-source toolchain designed specifically for cryptocurrency quantitative trading and machine learning. It addresses pain points such as inconsistent formats, fine granularity, and lack of derived indicators in Binance's raw data. It provides data downloading, aggregation, cleaning, and analysis capabilities, including core functions like batch acquisition, custom aggregation, technical indicator calculation, and whale detection, helping developers process data efficiently.

In the field of cryptocurrency quantitative trading and machine learning, data quality determines the success or failure of strategies. Binance's raw data has issues like inconsistent formats, overly fine granularity, and lack of derived indicators. Manual processing is time-consuming and error-prone. AggTradesTool was born to address this, providing one-stop data acquisition, cleaning, aggregation, and analysis capabilities.

AggTradesTool core features include:

1. Batch Download of Historical Data: Efficiently obtain aggregated trade data for any time period from Binance API, automatically handling pagination and rate limits;
2. Data Aggregation and Resampling: Generate OHLCV data according to custom time windows (e.g., 1 minute, 5 minutes);
3. Advanced Indicator Calculation: Built-in technical indicators like moving averages, RSI, MACD;
4. Whale Trade Detection: Identify large transactions via thresholds to spot abnormal market signals.

Data Acquisition Layer

Uses asynchronous requests to optimize download efficiency, connection pool + request queue management to avoid API limits, and supports resuming interrupted downloads of historical data.

Data Cleaning and Conversion

Automatically handles time zone conversion, missing value filling, and abnormal transaction filtering to ensure data reliability.

Whale Detection Algorithm

Users configure amount thresholds (e.g., over 10 BTC/100 ETH), the system marks large transactions and counts their distribution across time windows, helping identify institutional fund movements.

Quantitative Trading Researchers

Standardized preprocessing流程 allows researchers to focus on strategy logic rather than data engineering;

Machine Learning Engineers

Technical indicators and whale signals can be used as input features for models to improve prediction accuracy;

Market Analysts

Whale detection quickly identifies abnormal activities; combining with other data sources helps find investment opportunities or risks.

The cryptocurrency market fluctuates 24/7; timely data processing is a competitive advantage. AggTradesTool reduces manual processing time to a few minutes, lowering the barrier to quantitative research. It also provides reproducible processes, ensuring experimental results are traceable and reproducible, which is crucial for academic and production environments.

AggTradesTool bridges the gap from raw data to machine learning features and is a practical infrastructure for the quantitative community. It is an open-source project worth learning for beginners in crypto quantitative trading. In the future, integrating DeFi and more exchange data sources will further enhance the tool's value.