End-to-End Stock Price Prediction System Based on LSTM: A Complete Practice from Data Collection to API Deployment

This article introduces an end-to-end stock price prediction project m4-ml-lstm-bv-price, which uses LSTM neural networks to predict stock closing prices. It covers data collection, model training and evaluation, REST API deployment, monitoring dashboard, and Dockerized deployment solutions, providing a production-grade system template for data scientists and developers.

Stock market prediction is a core challenge in quantitative finance. Traditional time series methods (e.g., ARIMA) struggle to handle non-linear and highly volatile financial data. LSTM has significant advantages in financial time series prediction due to its gating mechanism that captures long-term dependencies. The project aims to provide a ready-to-run production-grade prediction system template, ensuring reproducibility and scalability of all components.

The data source is Yahoo Finance, and historical stock price data from 2018 to 2024 is obtained via the yfinance library (default ticker DIS, configurable to other codes). Preprocessing steps include: normalization using MinMaxScaler, constructing 60-day sliding windows as input features, splitting into training/validation/test sets, and saving in .npz format for model training.

The model uses a classic LSTM architecture (50 hidden units), and the training configuration includes an early stopping mechanism (max 100 epochs, batch size 32) to prevent overfitting. Evaluation uses MAE, RMSE, and MAPE metrics to measure prediction accuracy from multiple dimensions.

The API service is built based on FastAPI, providing two prediction interfaces: one that directly receives a price sequence of 60+ days and returns a prediction value, and another that only requires a stock ticker to automatically query historical data for prediction. The API includes health check and Prometheus metric endpoints, and integrates Prometheus (collects performance metrics), Grafana (visualization dashboard), and Streamlit (interactive terminal interface) for monitoring.

Deployment supports local development (Python virtual environment + Makefile to simplify the process) and Docker production mode (docker compose to start API, Prometheus, and Grafana with one click). Engineering highlights include modular design, .env configuration management, pytest unit tests, and comprehensive documentation to lower the entry barrier.

This project is an educational implementation, and model outputs should not be directly used as investment advice. The financial market is highly uncertain; data drift, market structure changes, and retraining mechanisms need to be considered. PyTorch is used to avoid AVX instruction set compatibility issues, adapting to more hardware environments.

This project provides a fully functional and clearly structured reference implementation for stock price prediction, demonstrating the application of LSTM in financial time series. It embodies modern machine learning engineering best practices (data pipeline, model serving, monitoring and alerting, containerized deployment) and is an important learning resource for developers to transform models into production systems.