Learning to Infer User Hidden States for Online Sequential Advertising
  To drive purchase in online advertising, it is of the advertiser's great
interest to optimize the sequential advertising strategy whose performance and
interpretability are both important. The lack of interpretability in existing
deep reinforcement learning methods makes it not easy to understand, diagnose
and further optimize the strategy. In this paper, we propose our Deep Intents
Sequential Advertising (DISA) method to address these issues. The key part of
interpretability is to understand a consumer's purchase intent which is,
however, unobservable (called hidden states). In this paper, we model this
intention as a latent variable and formulate the problem as a Partially
Observable Markov Decision Process (POMDP) where the underlying intents are
inferred based on the observable behaviors. Large-scale industrial offline and
online experiments demonstrate our method's superior performance over several
baselines. The inferred hidden states are analyzed, and the results prove the
rationality of our inference.
