Baseline Model Evaluation for Financial Time Series Forecasting: A Comparative Empirical Study of Machine Learning and Deep Learning Methods

This project aims to systematically evaluate the performance of various baseline machine learning (ML) and deep learning (DL) models on financial time series forecasting tasks, providing empirical references for model selection in quantitative trading and risk management. Key content includes the background of challenges in financial forecasting, evaluation methodology, introduction to various baseline models, interpretation of performance metrics, and model selection recommendations, helping readers understand the applicable scenarios and limitations of different models.

Stock price forecasting is extremely challenging due to the highly non-linear, non-stationary, and noisy characteristics of financial time series. The Efficient Market Hypothesis suggests that asset prices reflect all available information, making it difficult to consistently obtain excess returns using historical data. However, advances in ML/DL technologies provide new tools to capture patterns that are hard to detect with traditional methods. Based on this background, this project provides a reference for practitioners through empirical evaluation of baseline models.

The core objective of the project is to empirically evaluate the performance of classic, widely used baseline models. The evaluation methodology steps include: 1. Data collection (historical stock price data such as open/close prices, trading volume, etc.); 2. Feature engineering (extracting technical indicators or constructing lag features); 3. Data splitting (dividing into training/validation/test sets in chronological order to avoid data leakage); 4. Model training and hyperparameter tuning; 5. Performance evaluation (calculating metrics like MSE, MAE, etc.).

The baseline ML models evaluated include:

• Linear Regression: The simplest model, providing a reference for the lower bound of performance;
• Support Vector Regression (SVR): Handles high-dimensional non-linear data, but training is slow on large-scale data;
• Random Forest: An ensemble of decision trees, captures non-linear interactions, and is less prone to overfitting;
• Gradient Boosting Trees (e.g., XGBoost): Serial ensemble that fits residuals, and performs excellently in structured data competitions.

The baseline DL models evaluated include:

• Multi-Layer Perceptron (MLP): A basic neural network that approximates continuous functions, serving as the starting point for complex architectures;
• Recurrent Neural Networks (LSTM/GRU): Designed specifically for sequences, with gating mechanisms to mitigate gradient vanishing and capture long-term dependencies;
• 1D-CNN: Uses sliding windows to extract local patterns, trains quickly, and is easy to parallelize.

Financial forecasting evaluation needs to combine multiple metrics:

• Regression metrics (MSE, MAE);
• Supplementary metrics (directional accuracy, return backtesting, Sharpe ratio, confusion matrix). Key pitfalls to note in result interpretation: overfitting, lookahead bias (future information leakage), survivorship bias (ignoring delisted stocks), and transaction costs (overly optimistic backtesting).

Model Selection Recommendations:

• Short-term forecasting (1-5 days): Sequence models like LSTM are better;
• Long-term trends: Random Forest/XGBoost are more robust;
• The quality of feature engineering is more important than model selection;
• Ensemble models are better than single models;
• No model can consistently generate profits. Limitations: Limited dataset size, no consideration of external factors, simple models, and lack of real-time validation. Future Directions: Attention mechanism models, multi-task learning, reinforcement learning optimization strategies, and integration of alternative data.

This project provides a solid baseline comparison benchmark for financial time series forecasting. Before pursuing cutting-edge models, evaluating classic methods is a responsible research attitude. This 'from simple to complex' exploration helps understand the essence of the problem and avoids resource waste due to over-engineering.