  Anatomical atlases are widely used for population analysis. Conditional
atlases target a particular sub-population defined via certain conditions (e.g.
demographics or pathologies) and allow for the investigation of fine-grained
anatomical differences - such as morphological changes correlated with age.
Existing approaches use either registration-based methods that are unable to
handle large anatomical variations or generative models, which can suffer from
training instabilities and hallucinations. To overcome these limitations, we
use latent diffusion models to generate deformation fields, which transform a
general population atlas into one representing a specific sub-population. By
generating a deformation field and registering the conditional atlas to a
neighbourhood of images, we ensure structural plausibility and avoid
hallucinations, which can occur during direct image synthesis. We compare our
method to several state-of-the-art atlas generation methods in experiments
using 5000 brain as well as whole-body MR images from UK Biobank. Our method
generates highly realistic atlases with smooth transformations and high
anatomical fidelity, outperforming the baselines.
