  Click-Through Rate prediction (CTR) is a crucial task in recommender systems,
and it gained considerable attention in the past few years. The primary purpose
of recent research emphasizes obtaining meaningful and powerful representations
through mining low and high feature interactions using various components such
as Deep Neural Networks (DNN), CrossNets, or transformer blocks. In this work,
we propose the Deep Multi-Representation model (DeepMR) that jointly trains a
mixture of two powerful feature representation learning components, namely DNNs
and multi-head self-attentions. Furthermore, DeepMR integrates the novel
residual with zero initialization (ReZero) connections to the DNN and the
multi-head self-attention components for learning superior input
representations. Experiments on three real-world datasets show that the
proposed model significantly outperforms all state-of-the-art models in the
task of click-through rate prediction.
