  In this work we present affinity-VAE: a framework for automatic clustering
and classification of objects in multidimensional image data based on their
similarity. The method expands on the concept of $\beta$-VAEs with an informed
similarity-based loss component driven by an affinity matrix. The affinity-VAE
is able to create rotationally-invariant, morphologically homogeneous clusters
in the latent representation, with improved cluster separation compared with a
standard $\beta$-VAE. We explore the extent of latent disentanglement and
continuity of the latent spaces on both 2D and 3D image data, including
simulated biological electron cryo-tomography (cryo-ET) volumes as an example
of a scientific application.
