  Neural networks powered with external memory simulate computer behaviors.
These models, which use the memory to store data for a neural controller, can
learn algorithms and other complex tasks. In this paper, we introduce a new
memory to store weights for the controller, analogous to the stored-program
memory in modern computer architectures. The proposed model, dubbed Neural
Stored-program Memory, augments current memory-augmented neural networks,
creating differentiable machines that can switch programs through time, adapt
to variable contexts and thus resemble the Universal Turing Machine. A wide
range of experiments demonstrate that the resulting machines not only excel in
classical algorithmic problems, but also have potential for compositional,
continual, few-shot learning and question-answering tasks.
