  Transfer learning is a recent field of machine learning research that aims to
resolve the challenge of dealing with insufficient training data in the domain
of interest. This is a particular issue with traditional deep neural networks
where a large amount of training data is needed. Recently, StochasticNets was
proposed to take advantage of sparse connectivity in order to decrease the
number of parameters that needs to be learned, which in turn may relax training
data size requirements. In this paper, we study the efficacy of transfer
learning on StochasticNet frameworks. Experimental results show ~7% improvement
on StochasticNet performance when the transfer learning is applied in training
step.
