  For numerous domains, including for instance earth observation, medical
imaging, astrophysics,..., available image and signal datasets often involve
irregular space-time sampling patterns and large missing data rates. These
sampling properties may be critical to apply state-of-the-art learning-based
(e.g., auto-encoders, CNNs,...), fully benefit from the available large-scale
observations and reach breakthroughs in the reconstruction and identification
of processes of interest. In this paper, we address the end-to-end learning of
representations of signals, images and image sequences from irregularly-sampled
data, i.e. when the training data involved missing data. From an analogy to
Bayesian formulation, we consider energy-based representations. Two energy
forms are investigated: one derived from auto-encoders and one relating to
Gibbs priors. The learning stage of these energy-based representations (or
priors) involve a joint interpolation issue, which amounts to solving an energy
minimization problem under observation constraints. Using a
neural-network-based implementation of the considered energy forms, we can
state an end-to-end learning scheme from irregularly-sampled data. We
demonstrate the relevance of the proposed representations for different
case-studies: namely, multivariate time series, 2D images and image sequences.
