rust-lstm: A Complete LSTM Neural Network Library Implemented in Rust

A complete LSTM neural network library implemented from scratch in Rust, supporting training, multiple optimizers, 12 learning rate schedulers, advanced regularization, as well as bidirectional LSTM and GRU variants.

• Original Author/Maintainer: SyntaxSpirits
• Source Platforms: GitHub / crates.io
• Original Title: rust-lstm
• Original Link: https://github.com/SyntaxSpirits/rust-lstm
• Crate URL: https://crates.io/crates/rust-lstm
• Documentation: https://docs.rs/rust-lstm
• Release Status: Continuously updated, current version v0.8
• License: MIT

---

rust-lstm is a complete LSTM (Long Short-Term Memory) neural network library implemented from scratch in Rust. Unlike calling PyTorch or TensorFlow in the Python ecosystem, this project demonstrates how to build deep learning infrastructure from scratch using a systems-level language.

For developers who want to understand the inner workings of neural networks, or engineers who need to integrate sequence modeling capabilities into Rust projects, this is an extremely valuable resource.

---

This library provides core components of modern deep learning frameworks:

• LSTM Network: Standard Long Short-Term Memory network, supporting multi-layer stacking
• Bidirectional LSTM (BiLSTM): Processes forward and backward sequences simultaneously, supporting multiple merging modes
• GRU Network: Gated Recurrent Unit, with fewer parameters and faster training
• Peephole LSTM: LSTM variant with peephole connections
• Linear Layer (Dense): Fully connected layer for classification and output projection

• BPTT: Backpropagation Through Time
• Batch Processing: Supports efficient batch operations
• Early Stopping: Configurable patience value and metric monitoring

• Optimizers: SGD (with momentum), Adam (with bias correction), RMSprop
• Learning Rate Schedulers: Up to 12 strategies
  • ConstantLR (Constant)
  • StepLR (Step Decay)
  • MultiStepLR (Multi-stage Decay)
  • ExponentialLR (Exponential Decay)
  • CosineAnnealingLR (Cosine Annealing)
  • CosineAnnealingWarmRestarts (Cosine Annealing with Warm Restarts)
  • OneCycleLR (One-Cycle Policy)
  • ReduceLROnPlateau (Adaptive Decay on Plateau)
  • LinearLR (Linear Interpolation)
  • PolynomialLR (Polynomial Decay)
  • CyclicalLR (Triangular Cycle)
  • WarmupScheduler (Warmup Wrapper)

• Input Dropout: Applied to inputs before gate computation
• Recurrent Dropout: Applied to hidden states, supporting variational dropout
• Output Dropout: Applied to layer outputs
• Zoneout: RNN-specific regularization that retains previous state