  A method that allows us to give a different treatment to any neuron inside
feedforward neural networks is presented. The algorithm has been implemented
with two very different learning methods: a standard Back-propagation (BP)
procedure and an evolutionary algorithm. First, we have demonstrated that the
EA training method converges faster and gives more accurate results than BP.
Then we have made a full analysis of the effects of turning off different
combinations of neurons after the training phase. We demonstrate that EA is
much more robust than BP for all the cases under study. Even in the case when
two hidden neurons are lost, EA training is still able to give good average
results. This difference implies that we must be very careful when pruning or
redundancy effects are being studied since the network performance when losing
neurons strongly depends on the training method. Moreover, the influence of the
individual inputs will also depend on the training algorithm. Since EA keeps a
good classification performance when units are lost, this method could be a
good way to simulate biological learning systems since they must be robust
against deficient neuron performance. Although biological systems are much more
complex than the simulations shown in this article, we propose that a smart
training strategy such as the one shown here could be considered as a first
protection against the losing of a certain number of neurons.
