  Living neural networks emerge through a process of growth and
self-organization that begins with a single cell and results in a brain, an
organized and functional computational device. Artificial neural networks,
however, rely on human-designed, hand-programmed architectures for their
remarkable performance. Can we develop artificial computational devices that
can grow and self-organize without human intervention? In this paper, we
propose a biologically inspired developmental algorithm that can 'grow' a
functional, layered neural network from a single initial cell. The algorithm
organizes inter-layer connections to construct a convolutional pooling layer, a
key constituent of convolutional neural networks (CNN's). Our approach is
inspired by the mechanisms employed by the early visual system to wire the
retina to the lateral geniculate nucleus (LGN), days before animals open their
eyes. The key ingredients for robust self-organization are an emergent
spontaneous spatiotemporal activity wave in the first layer and a local
learning rule in the second layer that 'learns' the underlying activity pattern
in the first layer. The algorithm is adaptable to a wide-range of input-layer
geometries, robust to malfunctioning units in the first layer, and so can be
used to successfully grow and self-organize pooling architectures of different
pool-sizes and shapes. The algorithm provides a primitive procedure for
constructing layered neural networks through growth and self-organization.
Broadly, our work shows that biologically inspired developmental algorithms can
be applied to autonomously grow functional 'brains' in-silico.
