  Neural Machine Translation models tend to perpetuate gender bias present in
their training data distribution. Context-aware models have been previously
suggested as a means to mitigate this type of bias. In this work, we examine
this claim by analysing in detail the translation of stereotypical professions
in English to German, and translation with non-informative context in Basque to
Spanish. Our results show that, although context-aware models can significantly
enhance translation accuracy for feminine terms, they can still maintain or
even amplify gender bias. These results highlight the need for more
fine-grained approaches to bias mitigation in Neural Machine Translation.
