  We propose an efficient inference method for switching nonlinear dynamical
systems. The key idea is to learn an inference network which can be used as a
proposal distribution for the continuous latent variables, while performing
exact marginalization of the discrete latent variables. This allows us to use
the reparameterization trick, and apply end-to-end training with stochastic
gradient descent. We show that the proposed method can successfully segment
time series data, including videos and 3D human pose, into meaningful
``regimes'' by using the piece-wise nonlinear dynamics.
