  We demonstrate that almost all non-parametric dimensionality reduction
methods can be expressed by a simple procedure: regularized loss minimization
plus singular value truncation. By distinguishing the role of the loss and
regularizer in such a process, we recover a factored perspective that reveals
some gaps in the current literature. Beyond identifying a useful new loss for
manifold unfolding, a key contribution is to derive new convex regularizers
that combine distance maximization with rank reduction. These regularizers can
be applied to any loss.
