  This paper presents a new bi-Lipschitz invertible neural network, the
BiLipNet, which has the ability to smoothly control both its Lipschitzness
(output sensitivity to input perturbations) and inverse Lipschitzness (input
distinguishability from different outputs). The second main contribution is a
new scalar-output network, the PLNet, which is a composition of a BiLipNet and
a quadratic potential. We show that PLNet satisfies the Polyak-Lojasiewicz
condition and can be applied to learn non-convex surrogate losses with a unique
and efficiently-computable global minimum. The central technical element in
these networks is a novel invertible residual layer with certified strong
monotonicity and Lipschitzness, which we compose with orthogonal layers to
build the BiLipNet. The certification of these properties is based on
incremental quadratic constraints, resulting in much tighter bounds than can be
achieved with spectral normalization. Moreover, we formulate the calculation of
the inverse of a BiLipNet -- and hence the minimum of a PLNet -- as a series of
three-operator splitting problems, for which fast algorithms can be applied.
