  Quality Estimation (QE) is an important component of the machine translation
workflow as it assesses the quality of the translated output without consulting
reference translations. In this paper, we discuss our submission to the WMT
2021 QE Shared Task. We participate in Task 2 sentence-level sub-task that
challenge participants to predict the HTER score for sentence-level
post-editing effort. Our proposed system is an ensemble of multilingual BERT
(mBERT)-based regression models, which are generated by fine-tuning on
different input settings. It demonstrates comparable performance with respect
to the Pearson's correlation and beats the baseline system in MAE/ RMSE for
several language pairs. In addition, we adapt our system for the zero-shot
setting by exploiting target language-relevant language pairs and
pseudo-reference translations.
