  Neuroimaging datasets keep growing in size to address increasingly complex
medical questions. However, even the largest datasets today alone are too small
for training complex models or for finding genome wide associations. A solution
is to grow the sample size by merging data across several datasets. However,
bias in datasets complicates this approach and includes additional sources of
variation in the data instead. In this work, we combine 15 large neuroimaging
datasets to study bias. First, we detect bias by demonstrating that scans can
be correctly assigned to a dataset with 73.3% accuracy. Next, we introduce
metrics to quantify the compatibility across datasets and to create embeddings
of neuroimaging sites. Finally, we incorporate the presence of bias for the
selection of a training set for predicting autism. For the quantification of
the dataset bias, we introduce two metrics: the Bhattacharyya distance between
datasets and the age prediction error. The presented embedding of neuroimaging
sites provides an interesting new visualization about the similarity of
different sites. This could be used to guide the merging of data sources, while
limiting the introduction of unwanted variation. Finally, we demonstrate a
clear performance increase when incorporating dataset bias for training set
selection in autism prediction. Overall, we believe that the growing amount of
neuroimaging data necessitates to incorporate data-driven methods for
quantifying dataset bias in future analyses.
