  Transfer learning is known to perform efficiently in many applications
empirically, yet limited literature reports the mechanism behind the scene.
This study establishes both formal derivations and heuristic analysis to
formulate the theory of transfer learning in deep learning. Our framework
utilizing layer variational analysis proves that the success of transfer
learning can be guaranteed with corresponding data conditions. Moreover, our
theoretical calculation yields intuitive interpretations towards the knowledge
transfer process. Subsequently, an alternative method for network-based
transfer learning is derived. The method shows an increase in efficiency and
accuracy for domain adaptation. It is particularly advantageous when new domain
data is sufficiently sparse during adaptation. Numerical experiments over
diverse tasks validated our theory and verified that our analytic expression
achieved better performance in domain adaptation than the gradient descent
method.
