  Neural language models (LMs) perform well on tasks that require sensitivity
to syntactic structure. Drawing on the syntactic priming paradigm from
psycholinguistics, we propose a novel technique to analyze the representations
that enable such success. By establishing a gradient similarity metric between
structures, this technique allows us to reconstruct the organization of the
LMs' syntactic representational space. We use this technique to demonstrate
that LSTM LMs' representations of different types of sentences with relative
clauses are organized hierarchically in a linguistically interpretable manner,
suggesting that the LMs track abstract properties of the sentence.
