  The goal of emotional brain state classification on functional MRI (fMRI)
data is to recognize brain activity patterns related to specific emotion tasks
performed by subjects during an experiment. Distinguishing emotional brain
states from other brain states using fMRI data has proven to be challenging due
to two factors: a difficulty to generate fast yet accurate predictions in short
time frames, and a difficulty to extract emotion features which generalize to
unseen subjects. To address these challenges, we conducted an experiment in
which 22 subjects viewed pictures designed to stimulate either negative,
neutral or rest emotional responses while their brain activity was measured
using fMRI. We then developed two distinct Convolution-based approaches to
decode emotional brain states using only spatial information from single,
minimally pre-processed (slice timing and realignment) fMRI volumes. In our
first approach, we trained a 1D Convolutional Network (84.9% accuracy; chance
level 33%) to classify 3 emotion conditions using One-way Analysis of Variance
(ANOVA) voxel selection combined with hyperalignment. In our second approach,
we trained a 3D ResNet-50 model (78.0% accuracy; chance level 50%) to classify
2 emotion conditions from single 3D fMRI volumes directly. Our Convolutional
and Residual classifiers successfully learned group-level emotion features and
could decode emotion conditions from fMRI volumes in milliseconds. These
approaches could potentially be used in brain computer interfaces and real-time
fMRI neurofeedback research.
