  Neural Machine Translation (NMT) methodologies have burgeoned from using
simple feed-forward architectures to the state of the art; viz. BERT model. The
use cases of NMT models have been broadened from just language translations to
conversational agents (chatbots), abstractive text summarization, image
captioning, etc. which have proved to be a gem in their respective
applications. This paper aims to study the major trends in Neural Machine
Translation, the state of the art models in the domain and a high level
comparison between them.
