  Transferability estimation is a fundamental problem in transfer learning to
predict how good the performance is when transferring a source model (or source
task) to a target task. With the guidance of transferability score, we can
efficiently select the highly transferable source models without performing the
real transfer in practice. Recent analytical transferability metrics are mainly
designed for image classification problem, and currently there is no specific
investigation for the transferability estimation of semantic segmentation task,
which is an essential problem in autonomous driving, medical image analysis,
etc. Consequently, we further extend the recent analytical transferability
metric OTCE (Optimal Transport based Conditional Entropy) score to the semantic
segmentation task. The challenge in applying the OTCE score is the high
dimensional segmentation output, which is difficult to find the optimal
coupling between so many pixels under an acceptable computation cost. Thus we
propose to randomly sample N pixels for computing OTCE score and take the
expectation over K repetitions as the final transferability score. Experimental
evaluation on Cityscapes, BDD100K and GTA5 datasets demonstrates that the OTCE
score highly correlates with the transfer performance.
