  We present a new architecture for storing and accessing entity mentions
during online text processing. While reading the text, entity references are
identified, and may be stored by either updating or overwriting a cell in a
fixed-length memory. The update operation implies coreference with the other
mentions that are stored in the same cell; the overwrite operation causes these
mentions to be forgotten. By encoding the memory operations as differentiable
gates, it is possible to train the model end-to-end, using both a supervised
anaphora resolution objective as well as a supplementary language modeling
objective. Evaluation on a dataset of pronoun-name anaphora demonstrates strong
performance with purely incremental text processing.
