  Training a Neural Network (NN) with lots of parameters or intricate
architectures creates undesired phenomena that complicate the optimization
process. To address this issue we propose a first modular approach to NN
design, wherein the NN is decomposed into a control module and several
functional modules, implementing primitive operations. We illustrate the
modular concept by comparing performances between a monolithic and a modular NN
on a list sorting problem and show the benefits in terms of training speed,
training stability and maintainability. We also discuss some questions that
arise in modular NNs.
