  We present a semi-parametric generative model for predicting anatomy of a
patient in subsequent scans following a single baseline image. Such predictive
modeling promises to facilitate novel analyses in both voxel-level studies and
longitudinal biomarker evaluation. We capture anatomical change through a
combination of population-wide regression and a non-parametric model of the
subject's health based on individual genetic and clinical indicators. In
contrast to classical correlation and longitudinal analysis, we focus on
predicting new observations from a single subject observation. We demonstrate
prediction of follow-up anatomical scans in the ADNI cohort, and illustrate a
novel analysis approach that compares a patient's scans to the predicted
subject-specific healthy anatomical trajectory. The code is available at
https://github.com/adalca/voxelorb.
