  In recent years, Generative Adversarial Networks (GAN) have emerged as a
powerful method for learning the mapping from noisy latent spaces to realistic
data samples in high-dimensional space. So far, the development and application
of GANs have been predominantly focused on spatial data such as images. In this
project, we aim at modeling of spatio-temporal sensor data instead, i.e.
dynamic data over time. The main goal is to encode temporal data into a global
and low-dimensional latent vector that captures the dynamics of the
spatio-temporal signal. To this end, we incorporate auto-regressive RNNs,
Wasserstein GAN loss, spectral norm weight constraints and a semi-supervised
learning scheme into InfoGAN, a method for retrieval of meaningful latents in
adversarial learning. To demonstrate the modeling capability of our method, we
encode full-body skeletal human motion from a large dataset representing 60
classes of daily activities, recorded in a multi-Kinect setup. Initial results
indicate competitive classification performance of the learned latent
representations, compared to direct CNN/RNN inference. In future work, we plan
to apply this method on a related problem in the medical domain, i.e. on
recovery of meaningful latents in gait analysis of patients with vertigo and
balance disorders.
