  Convergence guarantees for optimization over bounded-rank matrices are
delicate to obtain because the feasible set is a non-smooth and non-convex
algebraic variety. Existing techniques include projected gradient descent,
fixed-rank optimization (over the maximal-rank stratum), and the LR
parameterization. They all lack either global guarantees (the ability to
accumulate only at critical points) or fast local convergence (e.g., if the
limit has non-maximal rank). We seek optimization algorithms that enjoy both.
  Khrulkov and Oseledets [2018] parameterize the bounded-rank variety via a
desingularization to recast the optimization problem onto a smooth manifold.
Building on their ideas, we develop a Riemannian geometry for this
desingularization, also with care for numerical considerations. We use it to
secure global convergence to critical points with fast local rates, for a large
range of algorithms. On matrix completion tasks, we find that this approach is
comparable to others, while enjoying better general-purpose theoretical
guarantees.
