  In the context of classification problems, Deep Learning (DL) approaches
represent state of art. Many DL approaches are based on variations of standard
multi-layer feed-forward neural networks. These are also referred to as deep
networks. The basic idea is that each hidden neural layer accomplishes a data
transformation which is expected to make the data representation "somewhat more
linearly separable" than the previous one to obtain a final data representation
which is as linearly separable as possible. However, determining the
appropriate neural network parameters that can perform these transformations is
a critical problem. In this paper, we investigate the impact on deep network
classifier performances of a training approach favouring solutions where data
representations at the hidden layers have a higher degree of linear
separability between the classes with respect to standard methods. To this aim,
we propose a neural network architecture which induces an error function
involving the outputs of all the network layers. Although similar approaches
have already been partially discussed in the past literature, here we propose a
new architecture with a novel error function and an extensive experimental
analysis. This experimental analysis was made in the context of image
classification tasks considering four widely used datasets. The results show
that our approach improves the accuracy on the test set in all the considered
cases.
