arrow
Volume 16, Issue 2
An Adaptive Physics-Informed Neural Network with Two-Stage Learning Strategy to Solve Partial Differential Equations

Shuyan Shi, Ding Liu, Ruirui Ji & Yuchao Han

Numer. Math. Theor. Meth. Appl., 16 (2023), pp. 298-322.

Published online: 2023-04

Export citation
  • Abstract

Physics-Informed Neural Network (PINN) represents a new approach to solve Partial Differential Equations (PDEs). PINNs aim to solve PDEs by integrating governing equations and the initial/boundary conditions (I/BCs) into a loss function. However, the imbalance of the loss function caused by parameter settings usually makes it difficult for PINNs to converge, e.g. because they fall into local optima. In other words, the presence of balanced PDE loss, initial loss and boundary loss may be critical for the convergence. In addition, existing PINNs are not able to reveal the hidden errors caused by non-convergent boundaries and conduction errors caused by the PDE near the boundaries. Overall, these problems have made PINN-based methods of limited use on practical situations. In this paper, we propose a novel physics-informed neural network, i.e. an adaptive physics-informed neural network with a two-stage training process. Our algorithm adds spatio-temporal coefficient and PDE balance parameter to the loss function, and solve PDEs using a two-stage training process: pre-training and formal training. The pre-training step ensures the convergence of boundary loss, whereas the formal training process completes the solution of PDE by balancing various loss functions. In order to verify the performance of our method, we consider the imbalanced heat conduction and Helmholtz equations often appearing in practical situations. The Klein-Gordon equation, which is widely used to compare performance, reveals that our method is able to reduce the hidden errors. Experimental results confirm that our algorithm can effectively and accurately solve models with unbalanced loss function, hidden errors and conduction errors. The codes developed in this manuscript are publicy available at https://github.com/callmedrcom/ATPINN.

  • AMS Subject Headings

35A25, 65N99, 65M99, 68T07

  • Copyright

COPYRIGHT: © Global Science Press

  • Email address
  • BibTex
  • RIS
  • TXT
@Article{NMTMA-16-298, author = {Shi , ShuyanLiu , DingJi , Ruirui and Han , Yuchao}, title = {An Adaptive Physics-Informed Neural Network with Two-Stage Learning Strategy to Solve Partial Differential Equations}, journal = {Numerical Mathematics: Theory, Methods and Applications}, year = {2023}, volume = {16}, number = {2}, pages = {298--322}, abstract = {

Physics-Informed Neural Network (PINN) represents a new approach to solve Partial Differential Equations (PDEs). PINNs aim to solve PDEs by integrating governing equations and the initial/boundary conditions (I/BCs) into a loss function. However, the imbalance of the loss function caused by parameter settings usually makes it difficult for PINNs to converge, e.g. because they fall into local optima. In other words, the presence of balanced PDE loss, initial loss and boundary loss may be critical for the convergence. In addition, existing PINNs are not able to reveal the hidden errors caused by non-convergent boundaries and conduction errors caused by the PDE near the boundaries. Overall, these problems have made PINN-based methods of limited use on practical situations. In this paper, we propose a novel physics-informed neural network, i.e. an adaptive physics-informed neural network with a two-stage training process. Our algorithm adds spatio-temporal coefficient and PDE balance parameter to the loss function, and solve PDEs using a two-stage training process: pre-training and formal training. The pre-training step ensures the convergence of boundary loss, whereas the formal training process completes the solution of PDE by balancing various loss functions. In order to verify the performance of our method, we consider the imbalanced heat conduction and Helmholtz equations often appearing in practical situations. The Klein-Gordon equation, which is widely used to compare performance, reveals that our method is able to reduce the hidden errors. Experimental results confirm that our algorithm can effectively and accurately solve models with unbalanced loss function, hidden errors and conduction errors. The codes developed in this manuscript are publicy available at https://github.com/callmedrcom/ATPINN.

}, issn = {2079-7338}, doi = {https://doi.org/10.4208/nmtma.OA-2022-0063}, url = {http://global-sci.org/intro/article_detail/nmtma/21578.html} }
TY - JOUR T1 - An Adaptive Physics-Informed Neural Network with Two-Stage Learning Strategy to Solve Partial Differential Equations AU - Shi , Shuyan AU - Liu , Ding AU - Ji , Ruirui AU - Han , Yuchao JO - Numerical Mathematics: Theory, Methods and Applications VL - 2 SP - 298 EP - 322 PY - 2023 DA - 2023/04 SN - 16 DO - http://doi.org/10.4208/nmtma.OA-2022-0063 UR - https://global-sci.org/intro/article_detail/nmtma/21578.html KW - Physics informed neural networks, partial differential equations, two-stage learning, scientific computing. AB -

Physics-Informed Neural Network (PINN) represents a new approach to solve Partial Differential Equations (PDEs). PINNs aim to solve PDEs by integrating governing equations and the initial/boundary conditions (I/BCs) into a loss function. However, the imbalance of the loss function caused by parameter settings usually makes it difficult for PINNs to converge, e.g. because they fall into local optima. In other words, the presence of balanced PDE loss, initial loss and boundary loss may be critical for the convergence. In addition, existing PINNs are not able to reveal the hidden errors caused by non-convergent boundaries and conduction errors caused by the PDE near the boundaries. Overall, these problems have made PINN-based methods of limited use on practical situations. In this paper, we propose a novel physics-informed neural network, i.e. an adaptive physics-informed neural network with a two-stage training process. Our algorithm adds spatio-temporal coefficient and PDE balance parameter to the loss function, and solve PDEs using a two-stage training process: pre-training and formal training. The pre-training step ensures the convergence of boundary loss, whereas the formal training process completes the solution of PDE by balancing various loss functions. In order to verify the performance of our method, we consider the imbalanced heat conduction and Helmholtz equations often appearing in practical situations. The Klein-Gordon equation, which is widely used to compare performance, reveals that our method is able to reduce the hidden errors. Experimental results confirm that our algorithm can effectively and accurately solve models with unbalanced loss function, hidden errors and conduction errors. The codes developed in this manuscript are publicy available at https://github.com/callmedrcom/ATPINN.

Shuyan Shi, Ding Liu, Ruirui Ji & Yuchao Han. (2023). An Adaptive Physics-Informed Neural Network with Two-Stage Learning Strategy to Solve Partial Differential Equations. Numerical Mathematics: Theory, Methods and Applications. 16 (2). 298-322. doi:10.4208/nmtma.OA-2022-0063
Copy to clipboard
The citation has been copied to your clipboard