  Can we obtain insights about the brain using AI models? How is the
information in deep learning models related to brain recordings? Can we improve
AI models with the help of brain recordings? Such questions can be tackled by
studying brain recordings like functional magnetic resonance imaging (fMRI). As
a first step, the neuroscience community has contributed several large
cognitive neuroscience datasets related to passive reading/listening/viewing of
concept words, narratives, pictures, and movies. Encoding and decoding models
using these datasets have also been proposed in the past two decades. These
models serve as additional tools for basic cognitive science and neuroscience
research. Encoding models aim at generating fMRI brain representations given a
stimulus automatically. They have several practical applications in evaluating
and diagnosing neurological conditions and thus may also help design therapies
for brain damage. Decoding models solve the inverse problem of reconstructing
the stimuli given the fMRI. They are useful for designing brain-machine or
brain-computer interfaces. Inspired by the effectiveness of deep learning
models for natural language processing, computer vision, and speech, several
neural encoding and decoding models have been recently proposed. In this
survey, we will first discuss popular representations of language, vision and
speech stimuli, and present a summary of neuroscience datasets. Further, we
will review popular deep learning based encoding and decoding architectures and
note their benefits and limitations. Finally, we will conclude with a summary
and discussion about future trends. Given the large amount of recently
published work in the computational cognitive neuroscience (CCN) community, we
believe that this survey enables an entry point for DNN researchers to
diversify into CCN research.
