  Humans can quickly associate stimuli to solve problems in novel contexts. Our
novel neural network model learns state representations of facts that can be
composed to perform such associative inference. To this end, we augment the
LSTM model with an associative memory, dubbed Fast Weight Memory (FWM). Through
differentiable operations at every step of a given input sequence, the LSTM
updates and maintains compositional associations stored in the rapidly changing
FWM weights. Our model is trained end-to-end by gradient descent and yields
excellent performance on compositional language reasoning problems,
meta-reinforcement-learning for POMDPs, and small-scale word-level language
modelling.
