  One of the challenges in model-based control of stochastic dynamical systems
is that the state transition dynamics are involved, and it is not easy or
efficient to make good-quality predictions of the states. Moreover, there are
not many representational models for the majority of autonomous systems, as it
is not easy to build a compact model that captures the entire dynamical
subtleties and uncertainties. In this work, we present a hierarchical Bayesian
linear regression model with local features to learn the dynamics of a
micro-robotic system as well as two simpler examples, consisting of a
stochastic mass-spring damper and a stochastic double inverted pendulum on a
cart. The model is hierarchical since we assume non-stationary priors for the
model parameters. These non-stationary priors make the model more flexible by
imposing priors on the priors of the model. To solve the maximum likelihood
(ML) problem for this hierarchical model, we use the variational expectation
maximization (EM) algorithm, and enhance the procedure by introducing hidden
target variables. The algorithm yields parsimonious model structures, and
consistently provides fast and accurate predictions for all our examples
involving large training and test sets. This demonstrates the effectiveness of
the method in learning stochastic dynamics, which makes it suitable for future
use in a paradigm, such as model-based reinforcement learning, to compute
optimal control policies in real time.
