  We present a novel deep learning framework that uses dynamic functional
connectivity to simultaneously localize the language and motor areas of the
eloquent cortex in brain tumor patients. Our method leverages convolutional
layers to extract graph-based features from the dynamic connectivity matrices
and a long-short term memory (LSTM) attention network to weight the relevant
time points during classification. The final stage of our model employs
multi-task learning to identify different eloquent subsystems. Our unique
training strategy finds a shared representation between the cognitive networks
of interest, which enables us to handle missing patient data. We evaluate our
method on resting-state fMRI data from 56 brain tumor patients while using task
fMRI activations as surrogate ground-truth labels for training and testing. Our
model achieves higher localization accuracies than conventional deep learning
approaches and can identify bilateral language areas even when trained on
left-hemisphere lateralized cases. Hence, our method may ultimately be useful
for preoperative mapping in tumor patients.
