  Automated image captioning is one of the applications of Deep Learning which
involves fusion of work done in computer vision and natural language
processing, and it is typically performed using Encoder-Decoder architectures.
In this project, we have implemented and experimented with various flavors of
multi-modal image captioning networks where ResNet101, DenseNet121 and VGG19
based CNN Encoders and Attention based LSTM Decoders were explored. We have
studied the effect of beam size and the use of pretrained word embeddings and
compared them to baseline CNN encoder and RNN decoder architecture. The goal is
to analyze the performance of each approach using various evaluation metrics
including BLEU, CIDEr, ROUGE and METEOR. We have also explored model
explainability using Visual Attention Maps (VAM) to highlight parts of the
images which has maximum contribution for predicting each word of the generated
caption.
