  Hamiltonian Monte Carlo (HMC) has emerged as a powerful Markov Chain Monte
Carlo (MCMC) method to sample from complex continuous distributions. However, a
fundamental limitation of HMC is that it can not be applied to distributions
with mixed discrete and continuous variables. In this paper, we propose mixed
HMC (M-HMC) as a general framework to address this limitation. M-HMC is a novel
family of MCMC algorithms that evolves the discrete and continuous variables in
tandem, allowing more frequent updates of discrete variables while maintaining
HMC's ability to suppress random-walk behavior. We establish M-HMC's
theoretical properties, and present an efficient implementation with Laplace
momentum that introduces minimal overhead compared to existing HMC methods. The
superior performances of M-HMC over existing methods are demonstrated with
numerical experiments on Gaussian mixture models (GMMs), variable selection in
Bayesian logistic regression (BLR), and correlated topic models (CTMs).
