  We study a class of Approximate Message Passing (AMP) algorithms for
symmetric and rectangular spiked random matrix models with orthogonally
invariant noise. The AMP iterates have fixed dimension $K \geq 1$, a
multivariate non-linearity is applied in each AMP iteration, and the algorithm
is spectrally initialized with $K$ super-critical sample eigenvectors. We
derive the forms of the Onsager debiasing coefficients and corresponding AMP
state evolution, which depend on the free cumulants of the noise spectral
distribution. This extends previous results for such models with $K=1$ and an
independent initialization.
  Applying this approach to Bayesian principal components analysis, we
introduce a Bayes-OAMP algorithm that uses as its non-linearity the posterior
mean conditional on all preceding AMP iterates. We describe a practical
implementation of this algorithm, where all debiasing and state evolution
parameters are estimated from the observed data, and we illustrate the accuracy
and stability of this approach in simulations.
