  In this paper, we address the issue of how to enhance the generalization
performance of convolutional neural networks (CNN) in the early learning stage
for image classification. This is motivated by real-time applications that
require the generalization performance of CNN to be satisfactory within limited
training time. In order to achieve this, a novel hierarchical transfer CNN
framework is proposed. It consists of a group of shallow CNNs and a cloud CNN,
where the shallow CNNs are trained firstly and then the first layers of the
trained shallow CNNs are used to initialize the first layer of the cloud CNN.
This method will boost the generalization performance of the cloud CNN
significantly, especially during the early stage of training. Experiments using
CIFAR-10 and ImageNet datasets are performed to examine the proposed method.
Results demonstrate the improvement of testing accuracy is 12% on average and
as much as 20% for the CIFAR-10 case while 5% testing accuracy improvement for
the ImageNet case during the early stage of learning. It is also shown that
universal improvements of testing accuracy are obtained across different
settings of dropout and number of shallow CNNs.
