  Irregularly-sampled time series occur in many domains including healthcare.
They can be challenging to model because they do not naturally yield a
fixed-dimensional representation as required by many standard machine learning
models. In this paper, we consider irregular sampling from the perspective of
missing data. We model observed irregularly-sampled time series data as a
sequence of index-value pairs sampled from a continuous but unobserved
function. We introduce an encoder-decoder framework for learning from such
generic indexed sequences. We propose learning methods for this framework based
on variational autoencoders and generative adversarial networks. For continuous
irregularly-sampled time series, we introduce continuous convolutional layers
that can efficiently interface with existing neural network architectures.
Experiments show that our models are able to achieve competitive or better
classification results on irregularly-sampled multivariate time series compared
to recent RNN models while offering significantly faster training times.
