  Click-Through Rate (CTR) prediction has become an essential task in digital
industries, such as digital advertising or online shopping. Many deep
learning-based methods have been implemented and have become state-of-the-art
models in the domain. To further improve the performance of CTR models,
Knowledge Distillation based approaches have been widely used. However, most of
the current CTR prediction models do not have much complex architectures, so
it's hard to call one of them 'cumbersome' and the other one 'tiny'. On the
other hand, the performance gap is also not very large between complex and
simple models. So, distilling knowledge from one model to the other could not
be worth the effort. Under these considerations, Mutual Learning could be a
better approach, since all the models could be improved mutually. In this
paper, we showed how useful the mutual learning algorithm could be when it is
between equals. In our experiments on the Criteo and Avazu datasets, the mutual
learning algorithm improved the performance of the model by up to 0.66%
relative improvement.
