  This paper proposes a novel nonlinear activation mechanism typically for
convolutional neural network (CNN), named as reborn mechanism. In sharp
contrast to ReLU which cuts off the negative phase value, the reborn mechanism
enjoys the capacity to reborn and reconstruct dead neurons. Compared to other
improved ReLU functions, reborn mechanism introduces a more proper way to
utilize the negative phase information. Extensive experiments validate that
this activation mechanism is able to enhance the model representation ability
more significantly and make the better use of the input data information while
maintaining the advantages of the original ReLU function. Moreover, reborn
mechanism enables a non-symmetry that is hardly achieved by traditional CNNs
and can act as a channel compensation method, offering competitive or even
better performance but with fewer learned parameters than traditional methods.
Reborn mechanism was tested on various benchmark datasets, all obtaining better
performance than previous nonlinear activation functions.
