  The application of deep learning (DL) models to the decoding of cognitive
states from whole-brain functional Magnetic Resonance Imaging (fMRI) data is
often hindered by the small sample size and high dimensionality of these
datasets. Especially, in clinical settings, where patient data are scarce. In
this work, we demonstrate that transfer learning represents a solution to this
problem. Particularly, we show that a DL model, which has been previously
trained on a large openly available fMRI dataset of the Human Connectome
Project, outperforms a model variant with the same architecture, but which is
trained from scratch, when both are applied to the data of a new, unrelated
fMRI task. Even further, the pre-trained DL model variant is already able to
correctly decode 67.51% of the cognitive states from a test dataset with 100
individuals, when fine-tuned on a dataset of the size of only three subjects.
