Solving Ordinary Differential Equations with Neural Networks: A Practical Exploration from NumPy to Surrogate Solvers

This project explores using neural networks to replace traditional numerical methods for solving ordinary differential equations (ODEs), providing efficient surrogate models for parameter scanning and real-time applications. Implemented purely in NumPy, the project balances educational transparency and engineering practicality, including a case study on the Logistic equation, multi-scenario application analysis, and future expansion directions.

Ordinary differential equations are widely used in natural sciences and engineering, but traditional numerical methods (e.g., Runge-Kutta) face bottlenecks in parameter scanning (requiring repeated integration) and real-time applications (high latency requirements). Neural networks can learn the mapping from parameters to solutions, requiring only one forward pass during inference, leading to significant speed improvements.

The project is implemented purely in NumPy and follows a three-stage process: 1. Data generation (sampling parameters, using traditional methods to compute exact solutions as training data); 2. Network training (fully connected network, input initial conditions and parameters, output solution vectors at fixed time points, optimized with mini-batch SGD); 3. Inference evaluation (quickly obtaining approximate solutions via forward propagation). The network is designed to predict the entire solution curve at once, differing from PINNs' point-by-point solving approach.

Taking the Logistic equation in population dynamics (dy/dt = ry(1-y/K)) as an example, a normalized version of the solver is implemented. The input includes the initial condition y₀ and growth rate r, and the output is the solution curve over the entire time interval. After training, the network can directly predict the complete curve from parameters without numerical integration.

The project's results can be applied to: 1. Real-time control systems (e.g., solving kinematic equations for robots/autonomous driving); 2. Parameter optimization (accelerating parameter search in engineering design); 3. Interactive visualization (instant feedback on system responses to parameter changes); 4. Embedded deployment (converting to ONNX/TFLite format for resource-constrained devices).

The author plans to expand: 1. Comparative implementation in PyTorch (verifying correctness, GPU acceleration); 2. Support for more ODE types (pendulum, Van der Pol oscillator, stiff systems, PDEs); 3. Exploration of hybrid methods (combining the advantages of neural networks and traditional numerical methods).

Compared to traditional methods, the neural network approach is faster in multi-parameter scenarios but has moderate accuracy; compared to PINNs, this project outputs the complete solution curve at once, suitable for different scenarios. The project's value lies in education (understanding neural network details), research (exploring the boundaries of surrogate models), and engineering (accelerating real-time/parameter scanning). It is recommended that developers start with the Logistic example, try modifying the network structure, or implement new ODE solvers.