  Video description is the automatic generation of natural language sentences
that describe the contents of a given video. It has applications in human-robot
interaction, helping the visually impaired and video subtitling. The past few
years have seen a surge of research in this area due to the unprecedented
success of deep learning in computer vision and natural language processing.
Numerous methods, datasets and evaluation metrics have been proposed in the
literature, calling the need for a comprehensive survey to focus research
efforts in this flourishing new direction. This paper fills the gap by
surveying the state of the art approaches with a focus on deep learning models;
comparing benchmark datasets in terms of their domains, number of classes, and
repository size; and identifying the pros and cons of various evaluation
metrics like SPICE, CIDEr, ROUGE, BLEU, METEOR, and WMD. Classical video
description approaches combined subject, object and verb detection with
template based language models to generate sentences. However, the release of
large datasets revealed that these methods can not cope with the diversity in
unconstrained open domain videos. Classical approaches were followed by a very
short era of statistical methods which were soon replaced with deep learning,
the current state of the art in video description. Our survey shows that
despite the fast-paced developments, video description research is still in its
infancy due to the following reasons. Analysis of video description models is
challenging because it is difficult to ascertain the contributions, towards
accuracy or errors, of the visual features and the adopted language model in
the final description. Existing datasets neither contain adequate visual
diversity nor complexity of linguistic structures. Finally, current evaluation
metrics ...
