  Reading comprehension models often overfit to nuances of training datasets
and fail at adversarial evaluation. Training with adversarially augmented
dataset improves robustness against those adversarial attacks but hurts
generalization of the models. In this work, we present several effective
adversaries and automated data augmentation policy search methods with the goal
of making reading comprehension models more robust to adversarial evaluation,
but also improving generalization to the source domain as well as new domains
and languages. We first propose three new methods for generating QA
adversaries, that introduce multiple points of confusion within the context,
show dependence on insertion location of the distractor, and reveal the
compounding effect of mixing adversarial strategies with syntactic and semantic
paraphrasing methods. Next, we find that augmenting the training datasets with
uniformly sampled adversaries improves robustness to the adversarial attacks
but leads to decline in performance on the original unaugmented dataset. We
address this issue via RL and more efficient Bayesian policy search methods for
automatically learning the best augmentation policy combinations of the
transformation probability for each adversary in a large search space. Using
these learned policies, we show that adversarial training can lead to
significant improvements in in-domain, out-of-domain, and cross-lingual
(German, Russian, Turkish) generalization.
