  Change Captioning is a task that aims to describe the difference between
images with natural language. Most existing methods treat this problem as a
difference judgment without the existence of distractors, such as viewpoint
changes. However, in practice, viewpoint changes happen often and can overwhelm
the semantic difference to be described. In this paper, we propose a novel
visual encoder to explicitly distinguish viewpoint changes from semantic
changes in the change captioning task. Moreover, we further simulate the
attention preference of humans and propose a novel reinforcement learning
process to fine-tune the attention directly with language evaluation rewards.
Extensive experimental results show that our method outperforms the
state-of-the-art approaches by a large margin in both Spot-the-Diff and
CLEVR-Change datasets.
