  Langevin diffusion (LD) is one of the main workhorses for sampling problems.
However, its convergence rate can be significantly reduced if the target
distribution is a mixture of multiple densities, especially when each component
concentrates around a different mode. Replica exchange Langevin diffusion
(ReLD) is a sampling method that can circumvent this issue. In particular, ReLD
adds another LD sampling a high-temperature version of the target density, and
exchange the locations of two LDs according to a Metropolis-Hasting type of
law. This approach can be further extended to multiple replica exchange
Langevin diffusion (mReLD), where $K$ additional LDs are added to sample
distributions at different temperatures and exchanges take place between
neighboring-temperature processes. While ReLD and mReLD have been used
extensively in statistical physics, molecular dynamics, and other applications,
there is little existing analysis on its convergence rate and choices of
temperatures. This paper closes these gaps assuming the target distribution is
a mixture of log-concave densities. We show ReLD can obtain constant or even
better convergence rates even when the density components of the mixture
concentrate around isolated modes. We also show using mReLD with $K$ additional
LDs can achieve the same result while the exchange frequency only needs to be
$(1/K)$-th power of the one in ReLD.
