  Efficient sampling from a high-dimensional Gaussian distribution is an old
but high-stake issue. Vanilla Cholesky samplers imply a computational cost and
memory requirements which can rapidly become prohibitive in high dimension. To
tackle these issues, multiple methods have been proposed from different
communities ranging from iterative numerical linear algebra to Markov chain
Monte Carlo (MCMC) approaches. Surprisingly, no complete review and comparison
of these methods have been conducted. This paper aims at reviewing all these
approaches by pointing out their differences, close relations, benefits and
limitations. In addition to this state of the art, this paper proposes a
unifying Gaussian simulation framework by deriving a stochastic counterpart of
the celebrated proximal point algorithm in optimization. This framework offers
a novel and unifying revisit of most of the existing MCMC approaches while
extending them. Guidelines to choose the appropriate Gaussian simulation method
for a given sampling problem in high dimension are proposed and illustrated
with numerical examples.
