  One of the hardest problems in the area of Natural Language Processing and
Artificial Intelligence is automatically generating language that is coherent
and understandable to humans. Teaching machines how to converse as humans do
falls under the broad umbrella of Natural Language Generation. Recent years
have seen unprecedented growth in the number of research articles published on
this subject in conferences and journals both by academic and industry
researchers. There have also been several workshops organized alongside
top-tier NLP conferences dedicated specifically to this problem. All this
activity makes it hard to clearly define the state of the field and reason
about its future directions. In this work, we provide an overview of this
important and thriving area, covering traditional approaches, statistical
approaches and also approaches that use deep neural networks. We provide a
comprehensive review towards building open domain dialogue systems, an
important application of natural language generation. We find that,
predominantly, the approaches for building dialogue systems use seq2seq or
language models architecture. Notably, we identify three important areas of
further research towards building more effective dialogue systems: 1)
incorporating larger context, including conversation context and world
knowledge; 2) adding personae or personality in the NLG system; and 3)
overcoming dull and generic responses that affect the quality of
system-produced responses. We provide pointers on how to tackle these open
problems through the use of cognitive architectures that mimic human language
understanding and generation capabilities.
