  A common challenge in regression is that for many problems, the degrees of
freedom required for a high-quality solution also allows for overfitting.
Regularization is a class of strategies that seek to restrict the range of
possible solutions so as to discourage overfitting while still enabling good
solutions, and different regularization strategies impose different types of
restrictions. In this paper, we present a multilevel regularization strategy
that constructs and trains a hierarchy of neural networks, each of which has
layers that are wider versions of the previous network's layers. We draw
intuition and techniques from the field of Algebraic Multigrid (AMG),
traditionally used for solving linear and nonlinear systems of equations, and
specifically adapt the Full Approximation Scheme (FAS) for nonlinear systems of
equations to the problem of deep learning. Training through V-cycles then
encourage the neural networks to build a hierarchical understanding of the
problem. We refer to this approach as \emph{multilevel-in-width} to distinguish
from prior multilevel works which hierarchically alter the depth of neural
networks. The resulting approach is a highly flexible framework that can be
applied to a variety of layer types, which we demonstrate with both
fully-connected and convolutional layers. We experimentally show with PDE
regression problems that our multilevel training approach is an effective
regularizer, improving the generalize performance of the neural networks
studied.
