  Starting from childhood, the human brain restructures and rewires throughout
life. Characterizing such complex brain development requires effective analysis
of longitudinal and multi-modal neuroimaging data. Here, we propose such an
analysis approach named Longitudinal Correlation Analysis (LCA). LCA couples
the data of two modalities by first reducing the input from each modality to a
latent representation based on autoencoders. A self-supervised strategy then
relates the two latent spaces by jointly disentangling two directions, one in
each space, such that the longitudinal changes in latent representations along
those directions are maximally correlated between modalities. We applied LCA to
analyze the longitudinal T1-weighted and diffusion-weighted MRIs of 679 youths
from the National Consortium on Alcohol and Neurodevelopment in Adolescence.
Unlike existing approaches that focus on either cross-sectional or single-modal
modeling, LCA successfully unraveled coupled macrostructural and
microstructural brain development from morphological and diffusivity features
extracted from the data. A retesting of LCA on raw 3D image volumes of those
subjects successfully replicated the findings from the feature-based analysis.
Lastly, the developmental effects revealed by LCA were inline with the current
understanding of maturational patterns of the adolescent brain.
