  Existing question answering methods often assume that the input content
(e.g., documents or videos) is always accessible to solve the task.
Alternatively, memory networks were introduced to mimic the human process of
incremental comprehension and compression of the information in a
fixed-capacity memory. However, these models only learn how to maintain memory
by backpropagating errors in the answers through the entire network. Instead,
it has been suggested that humans have effective mechanisms to boost their
memorization capacities, such as rehearsal and anticipation. Drawing
inspiration from these, we propose a memory model that performs rehearsal and
anticipation while processing inputs to memorize important information for
solving question answering tasks from streaming data. The proposed mechanisms
are applied self-supervised during training through masked modeling tasks
focused on coreference information. We validate our model on a short-sequence
(bAbI) dataset as well as large-sequence textual (NarrativeQA) and video
(ActivityNet-QA) question answering datasets, where it achieves substantial
improvements over previous memory network approaches. Furthermore, our ablation
study confirms the proposed mechanisms' importance for memory models.
