Core Guide to the Neural Network Tournament Forecasting Framework

This project is Neural-Network-Tournament-Forecaster released by Muzamil4266 on GitHub on June 7, 2026 (Original Link). Its core is a machine learning tournament framework that automatically trains, evaluates, benchmarks, and ranks competing neural network architectures through a three-stage assessment process: training, mock exam, and final exam. It aims to solve the problem of model generalization ability in financial time series forecasting, replacing subjective architecture selection with data-driven competition.

Project Background and Motivation

A long-standing issue in the machine learning field: researchers overfocus on the model training process while ignoring the generalization ability on truly unseen data. This is particularly prominent in the financial forecasting field—stock market data is noisy and subject to institutional changes, which easily leads to model overfitting.

The core idea of this project: Let multiple candidate neural network architectures compete fairly in a structured tournament, with the winner determined by objective performance data rather than subjective architecture selection.

Data Strategy and Preparation

Three-Stage Dataset Design

1. Training Dataset: Used for model learning; all candidate architectures share this data.
2. Mock Exam Dataset: Independent evaluation stage with no training, eliminating overfitted architectures.
3. Final Exam Dataset: Ultimate evaluation, only mock exam winners participate, providing an estimate of real future performance.

Data Processing

• Rolling Window: Each sample uses the previous 90 trading cycles as input and the next 30 as the prediction target.
• Normalization: Based on anchor price—each value in the sequence is divided by the last observation of the 90-day lookback window to solve the problem of price differences between different stocks.

Tournament Architecture and Model Details

Competing Architectures

The framework simultaneously evaluates 10 neural networks with different layer configurations and neuron structures, including: deep compression architecture, symmetric expansion-compression architecture, progressive narrowing architecture, high-capacity dense network, experimental hybrid structure, etc. Each represents a different hypothesis about information flow.

Model Construction

Automatically converted to TensorFlow networks:

• Dense layers
• ReLU activation function
• Adam optimizer
• MAE loss function
• Output layer generates 30-step predictions (not single points).

Smart Caching

After generating dataset windows, input/target arrays are saved as NumPy files and directly loaded in subsequent runs to reduce preprocessing time.

Evaluation and Selection Process

Mock Exam Stage

After training, models are tested on unseen mock datasets, with performance measured by direction accuracy (correctly predicting market direction rather than exact prices), eliminating poorly performing architectures.

Final Selection and Final Exam

The best-performing architectures from the mock exam enter the final exam, evaluated using completely unseen data. Metrics include:

• Direction accuracy
• Mean Absolute Error (MAE)
• Root Mean Square Error (RMSE)
• Correlation (consistency between predicted and actual trends)

Ranked based on final metrics, the top performer is the tournament champion.

Conclusions and Practical Implications

Core Value

Turning model selection from a subjective decision into a data-driven competition process is crucial for high-risk fields like financial forecasting.

Key Insight

Truly reliable models are not those that perform best during training, but those with the strongest generalization ability on unseen data.

Application Extension

This tournament-style evaluation framework is reusable and applicable to time series forecasting tasks in other fields.