  Transfer learning from huge natural image datasets, fine-tuning of deep
neural networks and the use of the corresponding pre-trained networks have
become de facto the core of art analysis applications. Nevertheless, the
effects of transfer learning are still poorly understood. In this paper, we
first use techniques for visualizing the network internal representations in
order to provide clues to the understanding of what the network has learned on
artistic images. Then, we provide a quantitative analysis of the changes
introduced by the learning process thanks to metrics in both the feature and
parameter spaces, as well as metrics computed on the set of maximal activation
images. These analyses are performed on several variations of the transfer
learning procedure. In particular, we observed that the network could
specialize some pre-trained filters to the new image modality and also that
higher layers tend to concentrate classes. Finally, we have shown that a double
fine-tuning involving a medium-size artistic dataset can improve the
classification on smaller datasets, even when the task changes.
