full
search.png

Search

close.png

Cancel

arrow
Volume 31, Issue 4
Convergence Rate Analysis for Deep Ritz Method

Chenguang Duan, Yuling Jiao, Yanming Lai, Dingwei Li, Xiliang Lu & Jerry Zhijian Yang

DOI: 10.4208/cicp.OA-2021-0195

Commun. Comput. Phys., 31 (2022), pp. 1020-1048.

Published online: 2022-03

Preview Purchase PDF 2440 48814

Cited by

google scholar semantic scholar
Export citation
  • Abstract

Using deep neural networks to solve PDEs has attracted a lot of attentions recently. However, why the deep learning method works is falling far behind its empirical success. In this paper, we provide a rigorous numerical analysis on deep Ritz method (DRM) [47] for second order elliptic equations with Neumann boundary conditions. We establish the first nonasymptotic convergence rate in $H^1$ norm for DRM using deep networks with ${\rm ReLU}^2$ activation functions. In addition to providing a theoretical justification of DRM, our study also shed light on how to set the hyperparameter of depth and width to achieve the desired convergence rate in terms of number of training samples. Technically, we derive bound on the approximation error of deep ${\rm ReLU}^2$ network in $C^1$ norm and bound on the Rademacher complexity of the non-Lipschitz composition of gradient norm and ${\rm ReLU}^2$ network, both of which are of independent interest.

  • Keywords

Deep Ritz method, deep neural networks, convergence rate analysis.

  • AMS Subject Headings

62G05, 65N12, 65N15, 68T07

  • Copyright

COPYRIGHT: © Global Science Press

  • Email address
  • BibTex
  • RIS
  • TXT
@Article{CiCP-31-1020, author = {Duan , ChenguangJiao , YulingLai , YanmingLi , DingweiLu , Xiliang and Yang , Jerry Zhijian}, title = {Convergence Rate Analysis for Deep Ritz Method}, journal = {Communications in Computational Physics}, year = {2022}, volume = {31}, number = {4}, pages = {1020--1048}, abstract = {

Using deep neural networks to solve PDEs has attracted a lot of attentions recently. However, why the deep learning method works is falling far behind its empirical success. In this paper, we provide a rigorous numerical analysis on deep Ritz method (DRM) [47] for second order elliptic equations with Neumann boundary conditions. We establish the first nonasymptotic convergence rate in $H^1$ norm for DRM using deep networks with ${\rm ReLU}^2$ activation functions. In addition to providing a theoretical justification of DRM, our study also shed light on how to set the hyperparameter of depth and width to achieve the desired convergence rate in terms of number of training samples. Technically, we derive bound on the approximation error of deep ${\rm ReLU}^2$ network in $C^1$ norm and bound on the Rademacher complexity of the non-Lipschitz composition of gradient norm and ${\rm ReLU}^2$ network, both of which are of independent interest.

}, issn = {1991-7120}, doi = {https://doi.org/10.4208/cicp.OA-2021-0195}, url = {http://global-sci.org/intro/article_detail/cicp/20375.html} }
TY - JOUR T1 - Convergence Rate Analysis for Deep Ritz Method AU - Duan , Chenguang AU - Jiao , Yuling AU - Lai , Yanming AU - Li , Dingwei AU - Lu , Xiliang AU - Yang , Jerry Zhijian JO - Communications in Computational Physics VL - 4 SP - 1020 EP - 1048 PY - 2022 DA - 2022/03 SN - 31 DO - http://doi.org/10.4208/cicp.OA-2021-0195 UR - https://global-sci.org/intro/article_detail/cicp/20375.html KW - Deep Ritz method, deep neural networks, convergence rate analysis. AB -

Using deep neural networks to solve PDEs has attracted a lot of attentions recently. However, why the deep learning method works is falling far behind its empirical success. In this paper, we provide a rigorous numerical analysis on deep Ritz method (DRM) [47] for second order elliptic equations with Neumann boundary conditions. We establish the first nonasymptotic convergence rate in $H^1$ norm for DRM using deep networks with ${\rm ReLU}^2$ activation functions. In addition to providing a theoretical justification of DRM, our study also shed light on how to set the hyperparameter of depth and width to achieve the desired convergence rate in terms of number of training samples. Technically, we derive bound on the approximation error of deep ${\rm ReLU}^2$ network in $C^1$ norm and bound on the Rademacher complexity of the non-Lipschitz composition of gradient norm and ${\rm ReLU}^2$ network, both of which are of independent interest.

Duan , ChenguangJiao , YulingLai , YanmingLi , DingweiLu , Xiliang and Yang , Jerry Zhijian. (2022). Convergence Rate Analysis for Deep Ritz Method. Communications in Computational Physics. 31 (4). 1020-1048. doi:10.4208/cicp.OA-2021-0195
Copy to clipboard
BibteX RIS TXT
The citation has been copied to your clipboard