  In this article, we propose a variational inference formulation of
auto-associative memories, allowing us to combine perceptual inference and
memory retrieval into the same mathematical framework. In this formulation, the
prior probability distribution onto latent representations is made memory
dependent, thus pulling the inference process towards previously stored
representations. We then study how different neural network approaches to
variational inference can be applied in this framework. We compare methods
relying on amortized inference such as Variational Auto Encoders and methods
relying on iterative inference such as Predictive Coding and suggest combining
both approaches to design new auto-associative memory models. We evaluate the
obtained algorithms on the CIFAR10 and CLEVR image datasets and compare them
with other associative memory models such as Hopfield Networks, End-to-End
Memory Networks and Neural Turing Machines.
