  "High Quality Related Search Query Suggestions" task aims at recommending
search queries which are real, accurate, diverse, relevant and engaging.
Obtaining large amounts of query-quality human annotations is expensive. Prior
work on supervised query suggestion models suffered from selection and exposure
bias, and relied on sparse and noisy immediate user-feedback (e.g., clicks),
leading to low quality suggestions. Reinforcement Learning techniques employed
to reformulate a query using terms from search results, have limited
scalability to large-scale industry applications. To recommend high quality
related search queries, we train a Deep Reinforcement Learning model to predict
the query a user would enter next. The reward signal is composed of long-term
session-based user feedback, syntactic relatedness and estimated naturalness of
generated query. Over the baseline supervised model, our proposed approach
achieves a significant relative improvement in terms of recommendation
diversity (3%), down-stream user-engagement (4.2%) and per-sentence word
repetitions (82%).
