  We present the contribution of the Unbabel team to the WMT 2019 Shared Task
on Quality Estimation. We participated on the word, sentence, and
document-level tracks, encompassing 3 language pairs: English-German,
English-Russian, and English-French. Our submissions build upon the recent
OpenKiwi framework: we combine linear, neural, and predictor-estimator systems
with new transfer learning approaches using BERT and XLM pre-trained models. We
compare systems individually and propose new ensemble techniques for word and
sentence-level predictions. We also propose a simple technique for converting
word labels into document-level predictions. Overall, our submitted systems
achieve the best results on all tracks and language pairs by a considerable
margin.
