  Disentanglement is the endeavour to use machine learning to divide
information about a dataset into meaningful fragments. In practice these
fragments are representation (sub)spaces, often the set of channels in the
latent space of a variational autoencoder (VAE). Assessments of disentanglement
predominantly employ metrics that are coarse-grained at the model level, but
this approach can obscure much about the process of information fragmentation.
Here we propose to study the learned channels in aggregate, as the fragments of
information learned by an ensemble of repeat training runs. Additionally, we
depart from prior work where measures of similarity between individual
subspaces neglected the nature of data embeddings as probability distributions.
Instead, we view representation subspaces as communication channels that
perform a soft clustering of the data; consequently, we generalize two classic
information-theoretic measures of similarity between clustering assignments to
compare representation spaces. We develop a lightweight method of estimation
based on fingerprinting representation subspaces by their ability to
distinguish dataset samples, allowing us to identify, analyze, and leverage
meaningful structure in ensembles of VAEs trained on synthetic and natural
datasets. Using this fully unsupervised pipeline we identify "hotspots" in the
space of information fragments: groups of nearly identical representation
subspaces that appear repeatedly in an ensemble of VAEs, particularly as
regularization is increased. Finally, we leverage the proposed methodology to
achieve ensemble learning with VAEs, boosting the information content of a set
of weak learners -- a capability not possible with previous methods of
assessing channel similarity.
