  What is the relationship between sentence representations learned by deep
recurrent models against those encoded by the brain? Is there any
correspondence between hidden layers of these recurrent models and brain
regions when processing sentences? Can these deep models be used to synthesize
brain data which can then be utilized in other extrinsic tasks? We investigate
these questions using sentences with simple syntax and semantics (e.g., The
bone was eaten by the dog.). We consider multiple neural network architectures,
including recently proposed ELMo and BERT. We use magnetoencephalography (MEG)
brain recording data collected from human subjects when they were reading these
simple sentences.
  Overall, we find that BERT's activations correlate the best with MEG brain
data. We also find that the deep network representation can be used to generate
brain data from new sentences to augment existing brain data.
  To the best of our knowledge, this is the first work showing that the MEG
brain recording when reading a word in a sentence can be used to distinguish
earlier words in the sentence. Our exploration is also the first to use deep
neural network representations to generate synthetic brain data and to show
that it helps in improving subsequent stimuli decoding task accuracy.
