PyStacks: A Modular CUDA Neural Network Library Built from Scratch

This article introduces PyStacks, a CUDA neural network library written entirely from scratch. The author uses TensorFlow and CuPy for GPU acceleration, adopts a modular design similar to Keras, and supports features like YOLO-style object detection, custom Concat graph optimization, and full training state saving. It is an excellent learning project for understanding the underlying principles of deep learning.

• Original Author/Maintainer: TheonlyIcebear
• Source Platform: GitHub
• Original Title: PyStacks
• Original Link: https://github.com/TheonlyIcebear/PyStacks
• Publication Date: 2026-05-25

---

Over the past year, the author has been continuously developing neural network projects, with the earliest project starting in April 2024. Through extensive trial-and-error learning, the final goal was to build a bounding box regression model similar to YOLO-v5.

The early code had many limitations and efficiency issues. Therefore, the author completely rewrote it using a modular design approach, similar to Keras' Sequential API. This architecture allows developers to quickly add new features (such as batch normalization, CSP blocks, multi-scale detection heads) without having to rewrite the entire system every time.

Important Note: This is not a replacement for TensorFlow or PyTorch. The author created it to understand what actually happens inside the "black box". If you value your time, do not use it in serious projects.

---

Build models by combining layers like Keras or PyTorch, but every forward pass, backward pass, and weight update is written manually—no automatic differentiation shortcuts.

Supports multi-scale anchor-based detection, including custom loss functions:

• DIoU, CIoU, SIoU: Improved bounding box regression losses
• Focal Loss: Addresses foreground-background class imbalance
• BCE: Binary cross-entropy classification loss
• Contraction Loss: Applies contraction loss to inactive anchors to prevent bounding box explosion during early training

Skip connections use the ConcatStartPoint / ConcatResidualStartPoint / ConcatEndPoint system to avoid storing redundant intermediate activation values. This is an optimization scheme independently designed by the author after the naive implementation consumed too much memory.

• Gradient accumulation
• AutoClipper: Adaptive gradient clipping (from original paper)
• Learning rate scheduler
• Full optimizer state saving and restoration

Uses TensorFlow (v1 compatibility mode) and CuPy for GPU computing, with manual memory management.