  Embed-to-control (E2C) is a model for solving high-dimensional optimal
control problems by combining variational auto-encoders with locally-optimal
controllers. However, the E2C model suffers from two major drawbacks: 1) its
objective function does not correspond to the likelihood of the data sequence
and 2) the variational encoder used for embedding typically has large
variational approximation error, especially when there is noise in the system
dynamics. In this paper, we present a new model for learning robust
locally-linear controllable embedding (RCE). Our model directly estimates the
predictive conditional density of the future observation given the current one,
while introducing the bottleneck between the current and future observations.
Although the bottleneck provides a natural embedding candidate for control, our
RCE model introduces additional specific structures in the generative graphical
model so that the model dynamics can be robustly linearized. We also propose a
principled variational approximation of the embedding posterior that takes the
future observation into account, and thus, makes the variational approximation
more robust against the noise. Experimental results show that RCE outperforms
the E2C model, and does so significantly when the underlying dynamics is noisy.
