  Efficient visual fault detection of freight trains is a critical part of
ensuring the safe operation of railways under the restricted hardware
environment. Although deep learning-based approaches have excelled in object
detection, the efficiency of freight train fault detection is still
insufficient to apply in real-world engineering. This paper proposes a
heterogeneous self-distillation framework to ensure detection accuracy and
speed while satisfying low resource requirements. The privileged information in
the output feature knowledge can be transferred from the teacher to the student
model through distillation to boost performance. We first adopt a lightweight
backbone to extract features and generate a new heterogeneous knowledge neck.
Such neck models positional information and long-range dependencies among
channels through parallel encoding to optimize feature extraction capabilities.
Then, we utilize the general distribution to obtain more credible and accurate
bounding box estimates. Finally, we employ a novel loss function that makes the
network easily concentrate on values near the label to improve learning
efficiency. Experiments on four fault datasets reveal that our framework can
achieve over 37 frames per second and maintain the highest accuracy in
comparison with traditional distillation approaches. Moreover, compared to
state-of-the-art methods, our framework demonstrates more competitive
performance with lower memory usage and the smallest model size.
