  The Teacher-Student (T-S) framework is widely utilized in the classification
tasks, through which the performance of one neural network (the student) can be
improved by transferring knowledge from another trained neural network (the
teacher). Since the transferring knowledge is related to the network capacities
and structures between the teacher and the student, how to define efficient
knowledge remains an open question. To address this issue, we design a novel
transferring knowledge, the Self-Attention based Inter-Class Correlation (ICC)
map in the output layer, and propose our T-S framework, Inter-Class Correlation
Transfer (ICCT).
