  Neural networks models for NLP are typically implemented without the explicit
encoding of language rules and yet they are able to break one performance
record after another. This has generated a lot of research interest in
interpreting the representations learned by these networks. We propose here a
novel interpretation approach that relies on the only processing system we have
that does understand language: the human brain. We use brain imaging recordings
of subjects reading complex natural text to interpret word and sequence
embeddings from 4 recent NLP models - ELMo, USE, BERT and Transformer-XL. We
study how their representations differ across layer depth, context length, and
attention type. Our results reveal differences in the context-related
representations across these models. Further, in the transformer models, we
find an interaction between layer depth and context length, and between layer
depth and attention type. We finally hypothesize that altering BERT to better
align with brain recordings would enable it to also better understand language.
Probing the altered BERT using syntactic NLP tasks reveals that the model with
increased brain-alignment outperforms the original model. Cognitive
neuroscientists have already begun using NLP networks to study the brain, and
this work closes the loop to allow the interaction between NLP and cognitive
neuroscience to be a true cross-pollination.
