  Deep learning builds deep architectures such as multi-layered artificial
neural networks to effectively represent multiple features of input patterns.
The adaptive structural learning method of Deep Belief Network (DBN) can
realize a high classification capability while searching the optimal network
structure during the training. The method can find the optimal number of hidden
neurons of a Restricted Boltzmann Machine (RBM) by neuron
generation-annihilation algorithm to train the given input data, and then it
can make a new layer in DBN by the layer generation algorithm to actualize a
deep data representation. Moreover, the learning algorithm of Adaptive RBM and
Adaptive DBN was extended to the time-series analysis by using the idea of LSTM
(Long Short Term Memory). In this paper, our proposed prediction method was
applied to Moving MNIST, which is a benchmark data set for video recognition.
We challenge to reveal the power of our proposed method in the video
recognition research field, since video includes rich source of visual
information. Compared with the LSTM model, our method showed higher prediction
performance (more than 90% predication accuracy for test data).
