  Extensive research has recently shown that recurrent neural language models
are able to process a wide range of grammatical phenomena. How these models are
able to perform these remarkable feats so well, however, is still an open
question. To gain more insight into what information LSTMs base their decisions
on, we propose a generalisation of Contextual Decomposition (GCD). In
particular, this setup enables us to accurately distil which part of a
prediction stems from semantic heuristics, which part truly emanates from
syntactic cues and which part arise from the model biases themselves instead.
We investigate this technique on tasks pertaining to syntactic agreement and
co-reference resolution and discover that the model strongly relies on a
default reasoning effect to perform these tasks.
