ePINN-AF: An Enhanced Physics-Informed Neural Network Fusing Attention Mechanism and Fuzzy Logic

Hello everyone! Today I'd like to introduce an enhanced physics-informed neural network project called ePINN-AF. Developed by amiin10, this project was released on GitHub (link: https://github.com/amiin10/ePINN-AF) on May 30, 2026. By integrating attention mechanisms and fuzzy logic, ePINN-AF aims to address the limitations of traditional physics-informed neural networks (PINNs) in multi-scale modeling, uncertainty quantification, and training stability, thereby improving their modeling capabilities in scientific computing and engineering problems. In the following floors, we will elaborate on the background, technical framework, application scenarios, and other content in detail.

Physics-Informed Neural Networks (PINNs) are a significant advancement in the field of scientific machine learning, proposed by Raissi et al. in 2019. Their core idea is to embed physical laws (such as partial differential equations) into the loss function, enabling the network to balance data learning and physical constraints. Core Advantages of PINNs:

• High data efficiency: Leveraging physical priors, they remain effective in data-scarce scenarios;
• Physical consistency: Ensures predictions comply with constraints like conservation laws, avoiding non-physical solutions;
• End-to-end solving: Supports forward (system evolution prediction) and inverse (parameter inference) problems. Limitations of PINNs:
• Multi-scale problems: A single network struggles to capture both fast and slow varying features;
• Training difficulties: Multiple competing terms in the loss function lead to gradient imbalance and slow convergence;
• Uncertainty quantification: Difficult to provide prediction uncertainty estimates, with limited robustness.

ePINN-AF (Enhanced PINN with Attention-Fuzzy Logic) addresses the limitations of PINNs by introducing two key technologies: Application of Attention Mechanism:

• Adaptive feature weighting: Dynamically focus on important regions and adjust network capacity;
• Multi-scale feature fusion: Capture macro and micro phenomena, improving the ability to capture features like boundary layers/shocks;
• Dynamic balance of physical constraints: Learn weights for each term in the loss function to enhance training stability. Application of Fuzzy Logic:
• Uncertainty quantification: Model data noise and uncertainty, outputting fuzzy sets or probability distributions;
• Fuzzy physical constraints: Allow approximate satisfaction of physical laws, balancing data and priors;
• Embedding expert knowledge: Formalize fuzzy language descriptions (e.g., "temperature is very high") to improve interpretability.

The enhanced capabilities of ePINN-AF are applicable to various complex scenarios:

• Fluid mechanics: Turbulence simulation (capturing multi-scale vortices), multiphase flow (fuzzy interface transition);
• Solid mechanics: Crack propagation (adaptive refinement of crack regions), composite materials (multi-scale modeling);
• Heat conduction: Phase change problems (dynamically tracking phase transition fronts);
• Biophysics: Neural signal conduction (focusing on key synaptic connections, handling system variability).

Advantages:

• Stronger expressive power: Attention mechanisms capture complex features, while fuzzy logic enriches the modeling language;
• Better uncertainty quantification: Fuzzy logic naturally handles uncertainty and outputs confidence levels;
• Higher training stability: Attention adjusts capacity, and fuzzy constraints make the optimization landscape smoother;
• Better interpretability: Attention weights can be visualized, and fuzzy rules express expert knowledge. Challenges:
• Computational complexity: Attention and fuzzy reasoning increase overhead;
• Hyperparameter tuning: The number of attention heads and fuzzy sets needs careful selection;
• Theoretical foundation: Systematic analysis of the convergence and generalization of the fusion framework is lacking.

By fusing attention mechanisms and fuzzy logic, ePINN-AF effectively enhances the modeling capabilities of PINNs, providing a new direction for solving multi-scale, uncertainty, and other problems. Related research trends include adaptive activation functions, domain-decomposed PINNs, Bayesian PINNs, graph neural network-based PINNs, etc. With the development of AI for Science, such cross-domain fusion architectures will open up more possibilities for scientific and engineering problems.