  Transfer learning enables solving a specific task having limited data by
using the pre-trained deep networks trained on large-scale datasets. Typically,
while transferring the learned knowledge from source task to the target task,
the last few layers are fine-tuned (re-trained) over the target dataset.
However, these layers are originally designed for the source task that might
not be suitable for the target task. In this paper, we introduce a mechanism
for automatically tuning the Convolutional Neural Networks (CNN) for improved
transfer learning. The pre-trained CNN layers are tuned with the knowledge from
target data using Bayesian Optimization. First, we train the final layer of the
base CNN model by replacing the number of neurons in the softmax layer with the
number of classes involved in the target task. Next, the pre-trained CNN is
tuned automatically by observing the classification performance on the
validation data (greedy criteria). To evaluate the performance of the proposed
method, experiments are conducted on three benchmark datasets, e.g.,
CalTech-101, CalTech-256, and Stanford Dogs. The classification results
obtained through the proposed AutoTune method outperforms the standard baseline
transfer learning methods over the three datasets by achieving $95.92\%$,
$86.54\%$, and $84.67\%$ accuracy over CalTech-101, CalTech-256, and Stanford
Dogs, respectively. The experimental results obtained in this study depict that
tuning of the pre-trained CNN layers with the knowledge from the target dataset
confesses better transfer learning ability. The source codes are available at
https://github.com/JekyllAndHyde8999/AutoTune_CNN_TransferLearning.
