  Contemporary data-driven methods are typically fed with full supervision on
large-scale datasets which limits their applicability. However, in the actual
systems with limitations such as measurement error and data acquisition
problems, people usually obtain incomplete data. Although data completion has
attracted wide attention, the underlying data pattern and relativity are still
under-developed. Currently, the family of latent variable models allows
learning deep latent variables over observed variables by fitting the marginal
distribution. As far as we know, current methods fail to perceive the data
relativity under partial observation. Aiming at modeling incomplete data, this
work uses relational inference to fill in the incomplete data. Specifically, we
expect to approximate the real joint distribution over the partial observation
and latent variables, thus infer the unseen targets respectively. To this end,
we propose Omni-Relational Network (OR-Net) to model the pointwise relativity
in two aspects: (i) On one hand, the inner relationship is built among the
context points in the partial observation; (ii) On the other hand, the unseen
targets are inferred by learning the cross-relationship with the observed data
points. It is further discovered that the proposed method can be generalized to
different scenarios regardless of whether the physical structure can be
observed or not. It is demonstrated that the proposed OR-Net can be well
generalized for data completion tasks of various modalities, including function
regression, image completion on MNIST and CelebA datasets, and also sequential
motion generation conditioned on the observed poses.
