  Pre-training a deep neural network on the ImageNet dataset is a common
practice for training deep learning models, and generally yields improved
performance and faster training times. The technique of pre-training on one
task and then retraining on a new one is called transfer learning. In this
paper we analyse the effectiveness of using deep transfer learning for
character recognition tasks. We perform three sets of experiments with varying
levels of similarity between source and target tasks to investigate the
behaviour of different types of knowledge transfer. We transfer both parameters
and features and analyse their behaviour. Our results demonstrate that no
significant advantage is gained by using a transfer learning approach over a
traditional machine learning approach for our character recognition tasks. This
suggests that using transfer learning does not necessarily presuppose a better
performing model in all cases.
