  We propose a scalable framework for the learning of high-dimensional
parametric maps via adaptively constructed residual network (ResNet) maps
between reduced bases of the inputs and outputs. When just few training data
are available, it is beneficial to have a compact parametrization in order to
ameliorate the ill-posedness of the neural network training problem. By
linearly restricting high-dimensional maps to informed reduced bases of the
inputs, one can compress high-dimensional maps in a constructive way that can
be used to detect appropriate basis ranks, equipped with rigorous error
estimates. A scalable neural network learning framework is thus to learn the
nonlinear compressed reduced basis mapping. Unlike the reduced basis
construction, however, neural network constructions are not guaranteed to
reduce errors by adding representation power, making it difficult to achieve
good practical performance. Inspired by recent approximation theory that
connects ResNets to sequential minimizing flows, we present an adaptive ResNet
construction algorithm. This algorithm allows for depth-wise enrichment of the
neural network approximation, in a manner that can achieve good practical
performance by first training a shallow network and then adapting. We prove
universal approximation of the associated neural network class for $L^2_\nu$
functions on compact sets. Our overall framework allows for constructive means
to detect appropriate breadth and depth, and related compact parametrizations
of neural networks, significantly reducing the need for architectural
hyperparameter tuning. Numerical experiments for parametric PDE problems and a
3D CFD wing design optimization parametric map demonstrate that the proposed
methodology can achieve remarkably high accuracy for limited training data, and
outperformed other neural network strategies we compared against.
