entropy-fold: Visualizing the Exact Geometric Structure of Piecewise Linear Neural Networks and the Catastrophe of Entropy Collapse

Introducing the entropy-fold project, which was published on GitHub by grewalsk on June 7, 2026. Implemented with pure NumPy and Matplotlib, its core objectives are to accurately visualize the piecewise linear geometric structure of ReLU neural networks and explore the catastrophe theory model of entropy collapse in reinforcement learning (RL) fine-tuning. Project link: https://github.com/grewalsk/entropy-fold.

The project originated from the need to deeply understand the internal mechanisms of neural networks rather than treating them as black boxes. Inspired by Jane Street's July 2024 technical blog Visualizing piecewise linear neural networks, the developer rebuilt core concepts from scratch to ensure full mastery. Key insight: ReLU neural networks are continuous piecewise linear functions—each ReLU activation acts like a hinge, cutting the input space into convex polygonal regions. Within each region, the network functions as an affine map, forming a polyhedral complex.

ReLU networks are essentially piecewise linear functions: a single ReLU divides the input space into two linear segments; inserting ReLU between linear layers preserves this property. The project uses LeakyReLU(0.02) to avoid local minima during the training of pure ReLU networks. Evolution of fold lines: For 2D input → 1D output, a single hidden layer has n fold lines (corresponding to neuron activation boundaries), and the number of feasible regions is 1 + n(n+1)/2 (theoretically 37 when n=8, with 32 visible in visualization). For multi-layer networks, fold lines bend across boundaries, and circular structures start to appear from the third layer.

1. Incremental region construction: Unlike Jane Street's method, an O(n²) incremental algorithm is used (initial bounding box → insert fold lines one by one → split only crossed cells using Sutherland-Hodgman clipping). It handles larger n values (875 regions for n=48 are completed instantly) but is only applicable to single-layer straight folds. 2. Zero-sampling error rendering: Evaluate at each real polygon vertex and draw planar polygons with no sampling error; color-code region identities and present an embossed effect by sorting by height.

The project's original intention was to visualize entropy collapse in GRPO (Group Relative Policy Optimization)—a phenomenon where the policy distribution sharpens to near-deterministic during RL fine-tuning of language models. This phenomenon is modeled as a cusp catastrophe (a sudden first-order jump caused by the folding of the equilibrium surface). fold3d.py renders the folded surface, and fold_app.py provides the 2D phase diagram. Connection: The policy network itself is piecewise linear, and entropy collapse is the process where local mappings become steep enough to saturate the softmax function.

Code Structure:

File	Description
foldnet.py	Implementation of the polyhedral complex for LeakyReLU network output surfaces
fold3d.py	3D rendering of the entropy collapse cusp surface
fold_app.py	2D phase diagram (shared color palette)
fold.py	Mathematical verification of entropy folding (self-test)

Usage:

1. Create and activate a virtual environment: python -m venv .venv && source .venv/bin/activate
2. Install dependencies: pip install -r requirements.txt
3. Run commands:
   • Basic run: python foldnet.py
   • High fold count: python foldnet.py --folds 32
   • Progression diagram: python foldnet.py --progression
   • Interactive 3D view: python foldnet.py --interactive

Technical value: 1. Pure NumPy + Matplotlib implementation of complex 3D visualization with no external dependencies; 2. Exact geometry outperforms approximate sampling; 3. Cross-domain connection between the piecewise linear geometry of neural networks and RL catastrophe theory; 4. The incremental algorithm reduces exponential complexity to polynomial level. Summary: entropy-fold is not just a visualization project but an exploration to deeply understand neural network mechanisms, providing an inspiring learning resource for developers and researchers.