Volume 1, Issue 3
The Cost-Accuracy Trade-Off in Operator Learning with Neural Networks

Maarten V. de Hoop, Daniel Zhengyu Huang, Elizabeth Qian & Andrew M. Stuart

J. Mach. Learn. , 1 (2022), pp. 299-341.

Published online: 2022-09

Primary category: Algorithm; Second category: Theory

Export citation
  • Abstract

The term ‘surrogate modeling’ in computational science and engineering refers to the development of computationally efficient approximations for expensive simulations, such as those arising from numerical solution of partial differential equations (PDEs). Surrogate modeling is an enabling methodology for many-query computations in science and engineering, which include iterative methods in optimization and sampling methods in uncertainty quantification. Over the last few years, several approaches to surrogate modeling for PDEs using neural networks have emerged, motivated by successes in using neural networks to approximate nonlinear maps in other areas. In principle, the relative merits of these different approaches can be evaluated by understanding, for each one, the cost required to achieve a given level of accuracy. However, the absence of a complete theory of approximation error for these approaches makes it difficult to assess this cost-accuracy trade-off. The purpose of the paper is to provide a careful numerical study of this issue, comparing a variety of different neural network architectures for operator approximation across a range of problems arising from PDE models in continuum mechanics.

  • General Summary

The term ‘surrogate modeling’ in computational science and engineering refers to the development of computationally efficient approximations for expensive simulations, such as those arising from numerical solutions of partial differential equations (PDEs). 

These surrogate models capture the behavior of these expensive models so that scientists and engineers can quickly study the dominant effects of the described system and use them for many-query computations, which include iterative methods in optimization and sampling methods in uncertainty quantification. 

Over the last few years, several approaches to surrogate modeling for PDEs using neural networks have emerged, motivated by successes in using neural networks to approximate nonlinear maps in other areas. In principle, the relative merits of these different approaches can be evaluated by understanding, for each one, the cost required to achieve a given level of accuracy. However, the absence of a complete theory of approximation error for these approaches makes it difficult to assess this cost-accuracy trade-off. The purpose of the paper is to provide a careful numerical study of this issue, comparing a variety of different neural network architectures for operator approximation across a range of problems arising from PDE models in continuum mechanics.

  • AMS Subject Headings

  • Copyright

COPYRIGHT: © Global Science Press

  • Email address
  • BibTex
  • RIS
  • TXT
@Article{JML-1-299, author = {Hoop , Maarten V. deHuang , Daniel ZhengyuQian , Elizabeth and Stuart , Andrew M.}, title = {The Cost-Accuracy Trade-Off in Operator Learning with Neural Networks}, journal = {Journal of Machine Learning}, year = {2022}, volume = {1}, number = {3}, pages = {299--341}, abstract = {

The term ‘surrogate modeling’ in computational science and engineering refers to the development of computationally efficient approximations for expensive simulations, such as those arising from numerical solution of partial differential equations (PDEs). Surrogate modeling is an enabling methodology for many-query computations in science and engineering, which include iterative methods in optimization and sampling methods in uncertainty quantification. Over the last few years, several approaches to surrogate modeling for PDEs using neural networks have emerged, motivated by successes in using neural networks to approximate nonlinear maps in other areas. In principle, the relative merits of these different approaches can be evaluated by understanding, for each one, the cost required to achieve a given level of accuracy. However, the absence of a complete theory of approximation error for these approaches makes it difficult to assess this cost-accuracy trade-off. The purpose of the paper is to provide a careful numerical study of this issue, comparing a variety of different neural network architectures for operator approximation across a range of problems arising from PDE models in continuum mechanics.

}, issn = {2790-2048}, doi = {https://doi.org/10.4208/jml.220509}, url = {http://global-sci.org/intro/article_detail/jml/21030.html} }
TY - JOUR T1 - The Cost-Accuracy Trade-Off in Operator Learning with Neural Networks AU - Hoop , Maarten V. de AU - Huang , Daniel Zhengyu AU - Qian , Elizabeth AU - Stuart , Andrew M. JO - Journal of Machine Learning VL - 3 SP - 299 EP - 341 PY - 2022 DA - 2022/09 SN - 1 DO - http://doi.org/10.4208/jml.220509 UR - https://global-sci.org/intro/article_detail/jml/21030.html KW - Computational partial differential equations, Surrogate modeling, Operator approximation, Neural networks, Computational complexity. AB -

The term ‘surrogate modeling’ in computational science and engineering refers to the development of computationally efficient approximations for expensive simulations, such as those arising from numerical solution of partial differential equations (PDEs). Surrogate modeling is an enabling methodology for many-query computations in science and engineering, which include iterative methods in optimization and sampling methods in uncertainty quantification. Over the last few years, several approaches to surrogate modeling for PDEs using neural networks have emerged, motivated by successes in using neural networks to approximate nonlinear maps in other areas. In principle, the relative merits of these different approaches can be evaluated by understanding, for each one, the cost required to achieve a given level of accuracy. However, the absence of a complete theory of approximation error for these approaches makes it difficult to assess this cost-accuracy trade-off. The purpose of the paper is to provide a careful numerical study of this issue, comparing a variety of different neural network architectures for operator approximation across a range of problems arising from PDE models in continuum mechanics.

Maarten V. de Hoop, Daniel Zhengyu Huang, Elizabeth Qian & Andrew M. Stuart. (2022). The Cost-Accuracy Trade-Off in Operator Learning with Neural Networks. Journal of Machine Learning. 1 (3). 299-341. doi:10.4208/jml.220509
Copy to clipboard
The citation has been copied to your clipboard