  Vision and language tasks have benefited from attention. There have been a
number of different attention models proposed. However, the scale at which
attention needs to be applied has not been well examined. Particularly, in this
work, we propose a new method Granular Multi-modal Attention, where we aim to
particularly address the question of the right granularity at which one needs
to attend while solving the Visual Dialog task. The proposed method shows
improvement in both image and text attention networks. We then propose a
granular Multi-modal Attention network that jointly attends on the image and
text granules and shows the best performance. With this work, we observe that
obtaining granular attention and doing exhaustive Multi-modal Attention appears
to be the best way to attend while solving visual dialog.
