  Neural machine translation (NMT) is often criticized for failures that happen
without awareness. The lack of competency awareness makes NMT untrustworthy.
This is in sharp contrast to human translators who give feedback or conduct
further investigations whenever they are in doubt about predictions. To fill
this gap, we propose a novel competency-aware NMT by extending conventional NMT
with a self-estimator, offering abilities to translate a source sentence and
estimate its competency. The self-estimator encodes the information of the
decoding procedure and then examines whether it can reconstruct the original
semantics of the source sentence. Experimental results on four translation
tasks demonstrate that the proposed method not only carries out translation
tasks intact but also delivers outstanding performance on quality estimation.
Without depending on any reference or annotated data typically required by
state-of-the-art metric and quality estimation methods, our model yields an
even higher correlation with human quality judgments than a variety of
aforementioned methods, such as BLEURT, COMET, and BERTScore. Quantitative and
qualitative analyses show better robustness of competency awareness in our
model.
