  This thesis investigates unsupervised time series representation learning for
sequence prediction problems, i.e. generating nice-looking input samples given
a previous history, for high dimensional input sequences by decoupling the
static input representation from the recurrent sequence representation. We
introduce three models based on Generative Stochastic Networks (GSN) for
unsupervised sequence learning and prediction. Experimental results for these
three models are presented on pixels of sequential handwritten digit (MNIST)
data, videos of low-resolution bouncing balls, and motion capture data. The
main contribution of this thesis is to provide evidence that GSNs are a viable
framework to learn useful representations of complex sequential input data, and
to suggest a new framework for deep generative models to learn complex
sequences by decoupling static input representations from dynamic time
dependency representations.
