  Language models have been shown to reproduce underlying biases existing in
their training data, which is the majority perspective by default. Proposed
solutions aim to capture minority perspectives by either modelling annotator
disagreements or grouping annotators based on shared metadata, both of which
face significant challenges. We propose a framework that trains models without
encoding annotator metadata, extracts latent embeddings informed by annotator
behaviour, and creates clusters of similar opinions, that we refer to as
voices. Resulting clusters are validated post-hoc via internal and external
quantitative metrics, as well a qualitative analysis to identify the type of
voice that each cluster represents. Our results demonstrate the strong
generalisation capability of our framework, indicated by resulting clusters
being adequately robust, while also capturing minority perspectives based on
different demographic factors throughout two distinct datasets.
