  There has been a recent surge of interest in the study of asymptotic
reconstruction performance in various cases of generalized linear estimation
problems in the teacher-student setting, especially for the case of i.i.d
standard normal matrices. Here, we go beyond these matrices, and prove an
analytical formula for the reconstruction performance of convex generalized
linear models with rotationally-invariant data matrices with arbitrary bounded
spectrum, rigorously confirming, under suitable assumptions, a conjecture
originally derived using the replica method from statistical physics. The proof
is achieved by leveraging on message passing algorithms and the statistical
properties of their iterates, allowing to characterize the asymptotic empirical
distribution of the estimator. For sufficiently strongly convex problems, we
show that the two-layer vector approximate message passing algorithm (2-MLVAMP)
converges, where the convergence analysis is done by checking the stability of
an equivalent dynamical system, which gives the result for such problems. We
then show that, under a concentration assumption, an analytical continuation
may be carried out to extend the result to convex (non-strongly) problems. We
illustrate our claim with numerical examples on mainstream learning methods
such as sparse logistic regression and linear support vector classifiers,
showing excellent agreement between moderate size simulation and the asymptotic
prediction.
