  State-space models (SSMs) are a highly expressive model class for learning
patterns in time series data and for system identification. Deterministic
versions of SSMs (e.g. LSTMs) proved extremely successful in modeling complex
time series data. Fully probabilistic SSMs, however, are often found hard to
train, even for smaller problems. To overcome this limitation, we propose a
novel model formulation and a scalable training algorithm based on doubly
stochastic variational inference and Gaussian processes. In contrast to
existing work, the proposed variational approximation allows one to fully
capture the latent state temporal correlations. These correlations are the key
to robust training. The effectiveness of the proposed PR-SSM is evaluated on a
set of real-world benchmark datasets in comparison to state-of-the-art
probabilistic model learning methods. Scalability and robustness are
demonstrated on a high dimensional problem.
