  In semantic parsing for question-answering, it is often too expensive to
collect gold parses or even gold answers as supervision signals. We propose to
convert model outputs into a set of human-understandable statements which allow
non-expert users to act as proofreaders, providing error markings as learning
signals to the parser. Because model outputs were suggested by a historic
system, we operate in a counterfactual, or off-policy, learning setup. We
introduce new estimators which can effectively leverage the given feedback and
which avoid known degeneracies in counterfactual learning, while still being
applicable to stochastic gradient optimization for neural semantic parsing.
Furthermore, we discuss how our feedback collection method can be seamlessly
integrated into deployed virtual personal assistants that embed a semantic
parser. Our work is the first to show that semantic parsers can be improved
significantly by counterfactual learning from logged human feedback data.
