PINNs-RL-PDE: Research on Adaptive Collocation Methods in Physics-Informed Neural Networks

This article introduces the PINNs-RL-PDE project, which integrates reinforcement learning (RL) into physics-informed neural networks (PINNs). To address the issues of low efficiency in collocation point selection and insufficient sampling in regions with sharp changes in solutions (such as shocks and boundary layers) in traditional PINNs, it optimizes adaptive collocation strategies through dynamic decision-making, improving the accuracy and efficiency of partial differential equation (PDE) solving.

Since their proposal in 2019, physics-informed neural networks (PINNs) have become an important method for PDE solving due to advantages like no need for mesh discretization, ease of handling high-dimensional problems, and integration of data and physical laws. However, traditional PINNs face key challenges in collocation point selection: uniform random sampling is inefficient, and there is insufficient sampling in regions where the solution changes sharply (e.g., shocks, boundary layers). The PINNs-RL-PDE project explores using reinforcement learning to automatically learn optimal collocation strategies.

The core idea is to model collocation point selection as a sequential decision-making problem: training is decomposed into an inner loop (training the network with fixed collocation points to minimize residuals) and an outer loop (RL adjusting collocation points). The state representation includes residual distribution, gradient information, historical collocation points, and training progress; the action space includes adding/removing collocation points, adjusting weights, etc.; the reward function integrates multiple objectives such as accuracy improvement, computational efficiency, convergence speed, and stability.

In technical implementation: optional RL algorithms include policy gradient, Actor-Critic, and PPO; integration with PINNs frameworks like DeepXDE is required, inserting collocation decision steps, updating collocation point sets, and calculating residuals; computational efficiency is optimized through experience replay, parallel sampling, and incremental updates.

Experiments were validated on classic PDEs such as the Burgers equation and Navier-Stokes equations. Results show: higher accuracy with the same number of collocation points, fewer collocation points needed for the same accuracy, automatic adaptation to local solution features (e.g., boundaries, shock regions), and excellent performance in complex geometry and multi-scale problems.

Theoretically, it discusses the conditions for the joint optimization framework to converge to the true solution; current limitations include: high complexity of joint training, unproven generalization ability of collocation strategies, high computational cost for high-dimensional problems, and sensitivity to RL hyperparameters.

Future directions include extending to multi-fidelity and multi-physics coupling, developing online learning and continuous adaptation systems, and deepening the theoretical foundation. This project demonstrates the prospects of combining RL with scientific machine learning, opening new avenues for improving PINNs' efficiency and robustness, and has far-reaching implications for computational physics and engineering simulation.