  In deep learning, transfer learning (TL) has become the de facto approach
when dealing with image related tasks. Visual features learnt for one task have
been shown to be reusable for other tasks, improving performance significantly.
By reusing deep representations, TL enables the use of deep models in domains
with limited data availability, limited computational resources and/or limited
access to human experts. Domains which include the vast majority of real-life
applications. This paper conducts an experimental evaluation of TL, exploring
its trade-offs with respect to performance, environmental footprint, human
hours and computational requirements. Results highlight the cases were a cheap
feature extraction approach is preferable, and the situations where an
expensive fine-tuning effort may be worth the added cost. Finally, a set of
guidelines on the use of TL are proposed.
