  We develop Microcanonical Hamiltonian Monte Carlo (MCHMC), a class of models
which follow a fixed energy Hamiltonian dynamics, in contrast to Hamiltonian
Monte Carlo (HMC), which follows canonical distribution with different energy
levels. MCHMC tunes the Hamiltonian function such that the marginal of the
uniform distribution on the constant-energy-surface over the momentum variables
gives the desired target distribution. We show that MCHMC requires occasional
energy conserving billiard-like momentum bounces for ergodicity, analogous to
momentum resampling in HMC. We generalize the concept of bounces to a
continuous version with partial direction preserving bounces at every step,
which gives an energy conserving underdamped Langevin-like dynamics with
non-Gaussian noise (MCLMC). MCHMC and MCLMC exhibit favorable scalings with
condition number and dimensionality. We develop an efficient hyperparameter
tuning scheme that achieves high performance and consistently outperforms NUTS
HMC on several standard benchmark problems, in some cases by more than an order
of magnitude.
