  We investigate the properties of local minima of the energy landscape of a
continuous non-convex optimization problem, the spherical perceptron with
piecewise linear cost function and show that they are critical, marginally
stable and displaying a set of pseudogaps, singularities and non-linear
excitations whose properties appear to be in the same universality class of
jammed packings of hard spheres. The piecewise linear perceptron problem
appears as an evolution of the purely linear perceptron optimization problem
that has been recently investigated in [1]. Its cost function contains two
non-analytic points where the derivative has a jump. Correspondingly, in the
non-convex/glassy phase, these two points give rise to four pseudogaps in the
force distribution and this induces four power laws in the gap distribution as
well. In addition one can define an extended notion of isostaticity and show
that local minima appear again to be isostatic in this phase. We believe that
our results generalize naturally to more complex cases with a proliferation of
non-linear excitations as the number of non-analytic points in the cost
function is increased.
