  Task transfer learning is a popular technique in image processing
applications that uses pre-trained models to reduce the supervision cost of
related tasks. An important question is to determine task transferability, i.e.
given a common input domain, estimating to what extent representations learned
from a source task can help in learning a target task. Typically,
transferability is either measured experimentally or inferred through task
relatedness, which is often defined without a clear operational meaning. In
this paper, we present a novel metric, H-score, an easily-computable evaluation
function that estimates the performance of transferred representations from one
task to another in classification problems using statistical and information
theoretic principles. Experiments on real image data show that our metric is
not only consistent with the empirical transferability measurement, but also
useful to practitioners in applications such as source model selection and task
transfer curriculum learning.
