  Artificial Neural Networks are uniquely adroit at machine learning by
processing data through a network of artificial neurons. The inter-neuronal
connection weights represent the learnt Neural Program that instructs the
network on how to compute the data. However, without an external memory to
store Neural Programs, they are restricted to only one, overwriting learnt
programs when trained on new data. This is functionally equivalent to a
special-purpose computer. Here we design Neurocoder, an entirely new class of
general-purpose conditional computational machines in which the neural network
"codes" itself in a data-responsive way by composing relevant programs from a
set of shareable, modular programs. This can be considered analogous to
building Lego structures from simple Lego bricks. Notably, our bricks change
their shape through learning. External memory is used to create, store and
retrieve modular programs. Like today's stored-program computers, Neurocoder
can now access diverse programs to process different data. Unlike manually
crafted computer programs, Neurocoder creates programs through training.
Integrating Neurocoder into current neural architectures, we demonstrate new
capacity to learn modular programs, handle severe pattern shifts and remember
old programs as new ones are learnt, and show substantial performance
improvement in solving object recognition, playing video games and continual
learning tasks. Such integration with Neurocoder increases the computation
capability of any current neural network and endows it with entirely new
capacity to reuse simple programs to build complex ones. For the first time a
Neural Program is treated as a datum in memory, paving the ways for modular,
recursive and procedural neural programming.
