  The term `surrogate modeling' in computational science and engineering refers
to the development of computationally efficient approximations for expensive
simulations, such as those arising from numerical solution of partial
differential equations (PDEs). Surrogate modeling is an enabling methodology
for many-query computations in science and engineering, which include iterative
methods in optimization and sampling methods in uncertainty quantification.
Over the last few years, several approaches to surrogate modeling for PDEs
using neural networks have emerged, motivated by successes in using neural
networks to approximate nonlinear maps in other areas. In principle, the
relative merits of these different approaches can be evaluated by
understanding, for each one, the cost required to achieve a given level of
accuracy. However, the absence of a complete theory of approximation error for
these approaches makes it difficult to assess this cost-accuracy trade-off. The
purpose of the paper is to provide a careful numerical study of this issue,
comparing a variety of different neural network architectures for operator
approximation across a range of problems arising from PDE models in continuum
mechanics.
