  Deep learning has revolutionized the computer vision and image classification
domains. In this context Convolutional Neural Networks (CNNs) based
architectures are the most widely applied models. In this article, we
introduced two procedures for training Convolutional Neural Networks (CNNs) and
Deep Neural Network based on Gradient Boosting (GB), namely GB-CNN and GB-DNN.
These models are trained to fit the gradient of the loss function or
pseudo-residuals of previous models. At each iteration, the proposed method
adds one dense layer to an exact copy of the previous deep NN model. The
weights of the dense layers trained on previous iterations are frozen to
prevent over-fitting, permitting the model to fit the new dense as well as to
fine-tune the convolutional layers (for GB-CNN) while still utilizing the
information already learned. Through extensive experimentation on different
2D-image classification and tabular datasets, the presented models show
superior performance in terms of classification accuracy with respect to
standard CNN and Deep-NN with the same architectures.
