  Modern semantic parsers suffer from two principal limitations. First,
training requires expensive collection of utterance-program pairs. Second,
semantic parsers fail to generalize at test time to new compositions/structures
that have not been observed during training. Recent research has shown that
automatic generation of synthetic utterance-program pairs can alleviate the
first problem, but its potential for the second has thus far been
under-explored. In this work, we investigate automatic generation of synthetic
utterance-program pairs for improving compositional generalization in semantic
parsing. Given a small training set of annotated examples and an "infinite"
pool of synthetic examples, we select a subset of synthetic examples that are
structurally-diverse and use them to improve compositional generalization. We
evaluate our approach on a new split of the schema2QA dataset, and show that it
leads to dramatic improvements in compositional generalization as well as
moderate improvements in the traditional i.i.d setup. Moreover,
structurally-diverse sampling achieves these improvements with as few as 5K
examples, compared to 1M examples when sampling uniformly at random -- a 200x
improvement in data efficiency.
