  Simulated tempering is popular method of allowing MCMC algorithms to move
between modes of a multimodal target density {\pi}. One problem with simulated
tempering for multimodal targets is that the weights of the various modes
change for different inverse-temperature values, sometimes dramatically so. In
this paper, we provide a fix to overcome this problem, by adjusting the mode
weights to be preserved (i.e., constant) over different inverse-temperature
settings. We then apply simulated tempering algorithms to multimodal targets
using our mode weight correction. We present simulations in which our
weight-preserving algorithm mixes between modes much more successfully than
traditional tempering algorithms. We also prove a diffusion limit for an
version of our algorithm, which shows that under appropriate assumptions, our
algorithm mixes in time O(d [log d]^2).
