  For text classification tasks, finetuned language models perform remarkably
well. Yet, they tend to rely on spurious patterns in training data, thus
limiting their performance on out-of-distribution (OOD) test data. Among recent
models aiming to avoid this spurious pattern problem, adding extra
counterfactual samples to the training data has proven to be very effective.
Yet, counterfactual data generation is costly since it relies on human
annotation. Thus, we propose a novel solution that only requires annotation of
a small fraction (e.g., 1%) of the original training data, and uses automatic
generation of extra counterfactuals in an encoding vector space. We demonstrate
the effectiveness of our approach in sentiment classification, using IMDb data
for training and other sets for OOD tests (i.e., Amazon, SemEval and Yelp). We
achieve noticeable accuracy improvements by adding only 1% manual
counterfactuals: +3% compared to adding +100% in-distribution training samples,
+1.3% compared to alternate counterfactual approaches.
