  Structural probing work has found evidence for latent syntactic information
in pre-trained language models. However, much of this analysis has focused on
monolingual models, and analyses of multilingual models have employed
correlational methods that are confounded by the choice of probing tasks. In
this study, we causally probe multilingual language models (XGLM and
multilingual BERT) as well as monolingual BERT-based models across various
languages; we do this by performing counterfactual perturbations on neuron
activations and observing the effect on models' subject-verb agreement
probabilities. We observe where in the model and to what extent syntactic
agreement is encoded in each language. We find significant neuron overlap
across languages in autoregressive multilingual language models, but not masked
language models. We also find two distinct layer-wise effect patterns and two
distinct sets of neurons used for syntactic agreement, depending on whether the
subject and verb are separated by other tokens. Finally, we find that
behavioral analyses of language models are likely underestimating how sensitive
masked language models are to syntactic information.
