  Describing what has changed in a scene can be useful to a user, but only if
generated text focuses on what is semantically relevant. It is thus important
to distinguish distractors (e.g. a viewpoint change) from relevant changes
(e.g. an object has moved). We present a novel Dual Dynamic Attention Model
(DUDA) to perform robust Change Captioning. Our model learns to distinguish
distractors from semantic changes, localize the changes via Dual Attention over
"before" and "after" images, and accurately describe them in natural language
via Dynamic Speaker, by adaptively focusing on the necessary visual inputs
(e.g. "before" or "after" image). To study the problem in depth, we collect a
CLEVR-Change dataset, built off the CLEVR engine, with 5 types of scene
changes. We benchmark a number of baselines on our dataset, and systematically
study different change types and robustness to distractors. We show the
superiority of our DUDA model in terms of both change captioning and
localization. We also show that our approach is general, obtaining
state-of-the-art results on the recent realistic Spot-the-Diff dataset which
has no distractors.
