  The deep linear network (DLN) is a model for implicit regularization in
gradient based optimization of overparametrized learning architectures.
Training the DLN corresponds to a Riemannian gradient flow, where the
Riemannian metric is defined by the architecture of the network and the loss
function is defined by the learning task. We extend this geometric framework,
obtaining explicit expressions for the volume form, including the case when the
network has infinite depth. We investigate the link between the Riemannian
geometry and the training asymptotics for matrix completion with rigorous
analysis and numerics. We propose that implicit regularization is a result of
bias towards high state space volume.
