  In light of the smoothness property brought by skip connections in ResNet,
this paper proposed the Skip Logit to introduce the skip connection mechanism
that fits arbitrary DNN dimensions and embraces similar properties to ResNet.
Meta Tanh Normalization (MTN) is designed to learn variance information and
stabilize the training process. With these delicate designs, our Skip Meta
Logit (SML) brought incremental boosts to the performance of extensive SOTA ctr
prediction models on two real-world datasets. In the meantime, we prove that
the optimization landscape of arbitrarily deep skip logit networks has no
spurious local optima. Finally, SML can be easily added to building blocks and
has delivered offline accuracy and online business metrics gains on app ads
learning to rank systems at TikTok.
