  The use of transfer learning with deep neural networks has increasingly
become widespread for deploying well-tested computer vision systems to newer
domains, especially those with limited datasets. We describe a transfer
learning use case for a domain with a data-starved regime, having fewer than
100 labeled target samples. We evaluate the effectiveness of convolutional
feature extraction and fine-tuning of overparameterized models with respect to
the size of target training data, as well as their generalization performance
on data with covariate shift, or out-of-distribution (OOD) data. Our
experiments demonstrate that both overparameterization and feature reuse
contribute to the successful application of transfer learning in training image
classifiers in data-starved regimes. We provide visual explanations to support
our findings and conclude that transfer learning enhances the performance of
CNN architectures in data-starved regimes.
