  Efficient crowd counting models are urgently required for the applications in
scenarios with limited computing resources, such as edge computing and mobile
devices. A straightforward method to achieve this is knowledge distillation
(KD), which involves using a trained teacher network to guide the training of a
student network. However, this traditional two-phase training method can be
time-consuming, particularly for large datasets, and it is also challenging for
the student network to mimic the learning process of the teacher network. To
overcome these challenges, we propose an online knowledge learning method for
crowd counting. Our method builds an end-to-end training framework that
integrates two independent networks into a single architecture, which consists
of a shared shallow module, a teacher branch, and a student branch. This
approach is more efficient than the two-stage training technique of traditional
KD. Moreover, we propose a feature relation distillation method which allows
the student branch to more effectively comprehend the evolution of inter-layer
features by constructing a new inter-layer relationship matrix. It is combined
with response distillation and feature internal distillation to enhance the
transfer of mutually complementary information from the teacher branch to the
student branch. Extensive experiments on four challenging crowd counting
datasets demonstrate the effectiveness of our method which achieves comparable
performance to state-of-the-art methods despite using far fewer parameters.
