  We study regularization in the context of small sample-size learning with
over-parameterized neural networks. Specifically, we shift focus from
architectural properties, such as norms on the network weights, to properties
of the internal representations before a linear classifier. Specifically, we
impose a topological constraint on samples drawn from the probability measure
induced in that space. This provably leads to mass concentration effects around
the representations of training instances, i.e., a property beneficial for
generalization. By leveraging previous work to impose topological constraints
in a neural network setting, we provide empirical evidence (across various
vision benchmarks) to support our claim for better generalization.
