Research on Hyperparameter Optimization of Fluid Dynamics Neural Networks: A Systematic Exploration of 648 Combinations

This study is an undergraduate thesis from the Department of Mathematics at TED University, focusing on hyperparameter optimization of fluid dynamics neural networks. It explores the application of mini-batch gradient descent in scientific computing through a systematic exploration of 648 hyperparameter combinations. The project provides dual implementations: PyTorch (automatic differentiation, GPU acceleration) and pure NumPy (transparent principles, teaching-friendly), and uses 5-fold cross-validation to evaluate model performance. The project source code is from the GitHub repository mbgd-fluid-dynamics-ann by user arincemir, released on July 12, 2026.

The core challenges of fluid dynamics stem from the nonlinearity, multi-scale nature, high dimensionality, and boundary complexity of the Navier-Stokes equations. Traditional Computational Fluid Dynamics (CFD) methods (finite difference, finite volume, etc.) are highly accurate but computationally expensive. The intervention of machine learning provides new tools for fluid dynamics: surrogate models to accelerate predictions, reduced-order models to simplify parametric studies, super-resolution to improve grid accuracy, inverse problem solving to infer boundary conditions, etc.

Technical Architecture: Dual implementations—PyTorch version uses automatic differentiation and GPU acceleration to simplify training; pure NumPy version implements forward/backward propagation from scratch, facilitating understanding of the algorithm's essence. Hyperparameter Search: Explores 648 combinations covering network architecture (number of hidden layers, number of neurons, activation functions), optimization parameters (learning rate, batch size, optimizer type), and regularization parameters (L2 strength, Dropout ratio). A grid search strategy and 5-fold cross-validation (3240 training runs) are used to ensure robust results.

The dataset consists of simulated fluid dynamics data generated by CFD software such as OpenFOAM, including input features (spatial coordinates, time steps, boundary conditions, material properties) and output targets (velocity components, pressure fields, vorticity). Preprocessing steps include normalization, dimensionality reduction, and spatiotemporal sampling to adapt to neural network training.

Hyperparameter Sensitivity: Learning rate is critical (too large leads to oscillation and divergence, too small results in slow convergence); batch size needs to balance efficiency and generalization (small batches have more noise but better generalization, large batches are faster but may have poor generalization); network depth must match task complexity (shallow networks underfit, deep networks are prone to overfitting). Optimal Configuration: The best configuration on the validation set is identified through evaluation using MSE, MAE, R², and physical constraint satisfaction. Result visualization includes comparisons between predicted fields and CFD results, error distribution analysis, and feature importance assessment.

Limitations: Simulated data is difficult to cover the complexity of real fluids, model generalization ability is limited, pure data-driven approaches may violate physical laws, and computational cost is high (648 ×5 training runs). Improvement Directions: Introduce Physics-Informed Neural Networks (PINNs) to incorporate Navier-Stokes equation constraints; use transfer learning to leverage simulated and real data; explore neural operators (e.g., FNO) to handle unstructured grids; use Bayesian optimization instead of grid search to improve efficiency.