  Unsupervised learning enables modeling complex images without the need for
annotations. The representation learned by such models can facilitate any
subsequent analysis of large image datasets.
  However, some generative factors that cause irrelevant variations in images
can potentially get entangled in such a learned representation causing the risk
of negatively affecting any subsequent use. The orientation of imaged objects,
for instance, is often arbitrary/irrelevant, thus it can be desired to learn a
representation in which the orientation information is disentangled from all
other factors.
  Here, we propose to extend the Variational Auto-Encoder framework by
leveraging the group structure of rotation-equivariant convolutional networks
to learn orientation-wise disentangled generative factors of histopathology
images. This way, we enforce a novel partitioning of the latent space, such
that oriented and isotropic components get separated.
  We evaluated this structured representation on a dataset that consists of
tissue regions for which nuclear pleomorphism and mitotic activity was assessed
by expert pathologists. We show that the trained models efficiently disentangle
the inherent orientation information of single-cell images. In comparison to
classical approaches, the resulting aggregated representation of
sub-populations of cells produces higher performances in subsequent tasks.
