  Recent advances have been made in applying convolutional neural networks to
achieve more precise prediction results for medical image segmentation
problems. However, the success of existing methods has highly relied on huge
computational complexity and massive storage, which is impractical in the
real-world scenario. To deal with this problem, we propose an efficient
architecture by distilling knowledge from well-trained medical image
segmentation networks to train another lightweight network. This architecture
empowers the lightweight network to get a significant improvement on
segmentation capability while retaining its runtime efficiency. We further
devise a novel distillation module tailored for medical image segmentation to
transfer semantic region information from teacher to student network. It forces
the student network to mimic the extent of difference of representations
calculated from different tissue regions. This module avoids the ambiguous
boundary problem encountered when dealing with medical imaging but instead
encodes the internal information of each semantic region for transferring.
Benefited from our module, the lightweight network could receive an improvement
of up to 32.6% in our experiment while maintaining its portability in the
inference phase. The entire structure has been verified on two widely accepted
public CT datasets LiTS17 and KiTS19. We demonstrate that a lightweight network
distilled by our method has non-negligible value in the scenario which requires
relatively high operating speed and low storage usage.
