  Understanding how the brain responds to sensory inputs is challenging: brain
recordings are partial, noisy, and high dimensional; they vary across sessions
and subjects and they capture highly nonlinear dynamics. These challenges have
led the community to develop a variety of preprocessing and analytical (almost
exclusively linear) methods, each designed to tackle one of these issues.
Instead, we propose to address these challenges through a specific end-to-end
deep learning architecture, trained to predict the brain responses of multiple
subjects at once. We successfully test this approach on a large cohort of
magnetoencephalography (MEG) recordings acquired during a one-hour reading
task. Our Deep Recurrent Encoding (DRE) architecture reliably predicts MEG
responses to words with a three-fold improvement over classic linear methods.
To overcome the notorious issue of interpretability of deep learning, we
describe a simple variable importance analysis. When applied to DRE, this
method recovers the expected evoked responses to word length and word
frequency. The quantitative improvement of the present deep learning approach
paves the way to better understand the nonlinear dynamics of brain activity
from large datasets.
