  We describe a parallel bayesian online deep learning framework (PBODL) for
click-through rate (CTR) prediction within today's Tencent advertising system,
which provides quick and accurate learning of user preferences. We first
explain the framework with a deep probit regression model, which is trained
with probabilistic back-propagation in the mode of assumed Gaussian density
filtering. Then we extend the model family to a variety of bayesian online
models with increasing feature embedding capabilities, such as Sparse-MLP,
FM-MLP and FFM-MLP. Finally, we implement a parallel training system based on a
stream computing infrastructure and parameter servers. Experiments with public
available datasets and Tencent industrial datasets show that models within our
framework perform better than several common online models, such as
AdPredictor, FTRL-Proximal and MatchBox. Online A/B test within Tencent
advertising system further proves that our framework could achieve CTR and CPM
lift by learning more quickly and accurately.
