  Quality Estimation (QE) models have the potential to change how we evaluate
and maybe even train machine translation models. However, these models still
lack the robustness to achieve general adoption. We show that State-of-the-art
QE models, when tested in a Parallel Corpus Mining (PCM) setting, perform
unexpectedly bad due to a lack of robustness to out-of-domain examples. We
propose a combination of multitask training, data augmentation and contrastive
learning to achieve better and more robust QE performance. We show that our
method improves QE performance significantly in the MLQE challenge and the
robustness of QE models when tested in the Parallel Corpus Mining setup. We
increase the accuracy in PCM by more than 0.80, making it on par with
state-of-the-art PCM methods that use millions of sentence pairs to train their
models. In comparison, we use a thousand times less data, 7K parallel sentences
in total, and propose a novel low resource PCM method.
