  In this paper, we revisit implicit regularization from the ground up using
notions from dynamical systems and invariant subspaces of Morse functions. The
key contributions are a new criterion for implicit regularization---a leading
contender to explain the generalization power of deep models such as neural
networks---and a general blueprint to study it. We apply these techniques to
settle a conjecture on implicit regularization in matrix factorization.
