  Many problems in NLP require aggregating information from multiple mentions
of the same entity which may be far apart in the text. Existing Recurrent
Neural Network (RNN) layers are biased towards short-term dependencies and
hence not suited to such tasks. We present a recurrent layer which is instead
biased towards coreferent dependencies. The layer uses coreference annotations
extracted from an external system to connect entity mentions belonging to the
same cluster. Incorporating this layer into a state-of-the-art reading
comprehension model improves performance on three datasets -- Wikihop, LAMBADA
and the bAbi AI tasks -- with large gains when training data is scarce.
