  We study the task of semantic parse correction with natural language
feedback. Given a natural language utterance, most semantic parsing systems
pose the problem as one-shot translation where the utterance is mapped to a
corresponding logical form. In this paper, we investigate a more interactive
scenario where humans can further interact with the system by providing
free-form natural language feedback to correct the system when it generates an
inaccurate interpretation of an initial utterance. We focus on natural language
to SQL systems and construct, SPLASH, a dataset of utterances, incorrect SQL
interpretations and the corresponding natural language feedback. We compare
various reference models for the correction task and show that incorporating
such a rich form of feedback can significantly improve the overall semantic
parsing accuracy while retaining the flexibility of natural language
interaction. While we estimated human correction accuracy is 81.5%, our best
model achieves only 25.1%, which leaves a large gap for improvement in future
research. SPLASH is publicly available at https://aka.ms/Splash_dataset.
