  Living organisms rely on internal models of the world to act adaptively.
These models cannot encode every detail and hence need to compress information.
From a cognitive standpoint, information compression can manifest as a
distortion of latent representations, resulting in the emergence of
representations that may not accurately reflect the external world or its
geometry. Rate-distortion theory formalizes the optimal way to compress
information, by considering factors such as capacity limitations, the frequency
and the utility of stimuli. However, while this theory explains why the above
factors distort latent representations, it does not specify which specific
distortions they produce. To address this question, here we systematically
explore the geometry of the latent representations that emerge in generative
models that operate under the principles of rate-distortion theory
($\beta$-VAEs). Our results highlight that three main classes of distortions of
internal representations -- prototypization, specialization, orthogonalization
-- emerge as signatures of information compression, under constraints on
capacity, data distributions and tasks. These distortions can coexist, giving
rise to a rich landscape of latent spaces, whose geometry could differ
significantly across generative models subject to different constraints. Our
findings contribute to explain how the normative constraints of rate-distortion
theory distort the geometry of latent representations of generative models of
artificial systems and living organisms.
