  Missingness is ubiquitous in multivariate time series and poses an obstacle
to reliable downstream analysis. Although recurrent network imputation achieved
the SOTA, existing models do not scale to deep architectures that can
potentially alleviate issues arising in complex data. Moreover, imputation
carries the risk of biased estimations of the ground truth. Yet, confidence in
the imputed values is always unmeasured or computed post hoc from model output.
We propose DEep Attention Recurrent Imputation (DEARI), which jointly estimates
missing values and their associated uncertainty in heterogeneous multivariate
time series. By jointly representing feature-wise correlations and temporal
dynamics, we adopt a self attention mechanism, along with an effective residual
component, to achieve a deep recurrent neural network with good imputation
performance and stable convergence. We also leverage self-supervised metric
learning to boost performance by optimizing sample similarity. Finally, we
transform DEARI into a Bayesian neural network through a novel Bayesian
marginalization strategy to produce stochastic DEARI, which outperforms its
deterministic equivalent. Experiments show that DEARI surpasses the SOTA in
diverse imputation tasks using real-world datasets, namely air quality control,
healthcare and traffic.
