  Task-oriented dialog (TOD) systems often need to formulate knowledge base
(KB) queries corresponding to the user intent and use the query results to
generate system responses. Existing approaches require dialog datasets to
explicitly annotate these KB queries -- these annotations can be time
consuming, and expensive. In response, we define the novel problems of
predicting the KB query and training the dialog agent, without explicit KB
query annotation. For query prediction, we propose a reinforcement learning
(RL) baseline, which rewards the generation of those queries whose KB results
cover the entities mentioned in subsequent dialog. Further analysis reveals
that correlation among query attributes in KB can significantly confuse memory
augmented policy optimization (MAPO), an existing state of the art RL agent. To
address this, we improve the MAPO baseline with simple but important
modifications suited to our task. To train the full TOD system for our setting,
we propose a pipelined approach: it independently predicts when to make a KB
query (query position predictor), then predicts a KB query at the predicted
position (query predictor), and uses the results of predicted query in
subsequent dialog (next response predictor). Overall, our work proposes first
solutions to our novel problem, and our analysis highlights the research
challenges in training TOD systems without query annotation.
