  Deep learning models have shown state-of-the-art performance in many inverse
reconstruction problems. However, it is not well understood what properties of
the latent representation may improve the generalization ability of the
network. Furthermore, limited models have been presented for inverse
reconstructions over time sequences. In this paper, we study the generalization
ability of a sequence encoder decoder model for solving inverse reconstructions
on time sequences. Our central hypothesis is that the generalization ability of
the network can be improved by 1) constrained stochasticity and 2) global
aggregation of temporal information in the latent space. First, drawing from
analytical learning theory, we theoretically show that a stochastic latent
space will lead to an improved generalization ability. Second, we consider an
LSTM encoder-decoder architecture that compresses a global latent vector from
all last-layer units in the LSTM encoder. This model is compared with
alternative LSTM encoder-decoder architectures, each in deterministic and
stochastic versions. The results demonstrate that the generalization ability of
an inverse reconstruction network can be improved by constrained stochasticity
combined with global aggregation of temporal information in the latent space.
