  Structural covariance analysis is a widely used structural MRI analysis
method which characterises the co-relations of morphology between brain regions
over a group of subjects. To our knowledge, little has been investigated in
terms of the comparability of results between different data sets or the
reliability of results over the same subjects in different rescan sessions,
image resolutions, or FreeSurfer versions.
  In terms of comparability, our results show substantial differences in the
structural covariance matrix between data sets of age- and sex-matched healthy
human adults. These differences persist after site correction, they are
exacerbated by low sample sizes, and they are most pronounced when using
average cortical thickness as a morphological measure. Down-stream graph
theoretic analyses further show statistically significant differences.
  In terms of reliability, substantial differences were also found when
comparing repeated scan sessions of the same subjects, and image resolutions
and FreeSurfer versions of the same image. We could further estimate the
relative measurement error and showed that it is largest when using thickness.
With simulated data, we argue that cortical thickness is least reliable because
of larger relative measurement errors.
  Practically, we make the following recommendations (1) pooling subjects
across sites into one group should be avoided, particularly if sites differ in
image resolutions, demographics, or preprocessing; (2) surface area and volume
should be preferred as morphological measures over cortical thickness; (3) a
large number of subjects should be used to estimate structural covariance; (4)
measurement error should be assessed where repeated measurements are available;
(5) if combining sites is critical, univariate site-correction is insufficient,
but error covariance should be explicitly measured and modelled.
