  We present a system, Spoke, for creating and searching internal knowledge
base (KB) articles for organizations. Spoke is available as a SaaS
(Software-as-a-Service) product deployed across hundreds of organizations with
a diverse set of domains. Spoke continually improves search quality using
conversational user feedback which allows it to provide better search
experience than standard information retrieval systems without encoding any
explicit domain knowledge. We achieve this by using a real-time online
learning-to-rank (L2R) algorithm that automatically customizes relevance
scoring for each organization deploying Spoke by using a query similarity
kernel.
  The focus of this paper is on incorporating practical considerations into our
relevance scoring function and algorithm that make Spoke easy to deploy and
suitable for handling events that naturally happen over the life-cycle of any
KB deployment. We show that Spoke outperforms competitive baselines by up to
41% in offline F1 comparisons.
