  This paper proposes a replica exchange preconditioned Langevin diffusion
discretized by the Crank-Nicolson scheme (repCNLD) to handle high-dimensional
and multi-modal distribution problems. Sampling from high-dimensional and
multi-modal distributions is a challenging question. The performance of many
standard MCMC chains deteriorates as the dimension of parameters increases, and
many MCMC algorithms cannot capture all modes if the energy function is not
convex. The proposed repCNLD can accelerate the convergence of the single-chain
pCNLD, and can capture all modes of the multi-modal distributions. We proposed
the Crank-Nicolson discretization, which is robust. Moreover, the
discretization error grows linearly with respect to the time step size. We
extend repCNLD to the multi-variance setting to further accelerate the
convergence and save computation costs. Additionally, we derive an unbiased
estimator of the swapping rate for the multi-variance repCNLD method, providing
a guide for the choice of the low-fidelity model used in the second chain. We
test our methods with high-dimensional Gaussian mixture models and
high-dimensional nonlinear PDE inverse problems. Particularly, we employ the
discrete adjoint method to efficiently calculate gradients for nonlinear PDE
inverse problems.
