  User response prediction makes a crucial contribution to the rapid
development of online advertising system and recommendation system. The
importance of learning feature interactions has been emphasized by many works.
Many deep models are proposed to automatically learn high-order feature
interactions. Since most features in advertising system and recommendation
system are high-dimensional sparse features, deep models usually learn a
low-dimensional distributed representation for each feature in the bottom
layer. Besides traditional fully-connected architectures, some new operations,
such as convolutional operations and product operations, are proposed to learn
feature interactions better. In these models, the representation is shared
among different operations. However, the best representation for different
operations may be different. In this paper, we propose a new neural model named
Operation-aware Neural Networks (ONN) which learns different representations
for different operations. Our experimental results on two large-scale
real-world ad click/conversion datasets demonstrate that ONN consistently
outperforms the state-of-the-art models in both offline-training environment
and online-training environment.
