  In many scientific problems such as video surveillance, modern genomics, and
finance, data are often collected from diverse measurements across time that
exhibit time-dependent heterogeneous properties. Thus, it is important to not
only integrate data from multiple sources (called multi-view data), but also to
incorporate time dependency for deep understanding of the underlying system. We
propose a generative model based on variational autoencoder and a recurrent
neural network to infer the latent dynamics for multi-view temporal data. This
approach allows us to identify the disentangled latent embeddings across views
while accounting for the time factor. We invoke our proposed model for
analyzing three datasets on which we demonstrate the effectiveness and the
interpretability of the model.
