  We propose a method for learning de-identified prosody representations from
raw audio using a contrastive self-supervised signal. Whereas prior work has
relied on conditioning models on bottlenecks, we introduce a set of inductive
biases that exploit the natural structure of prosody to minimize timbral
information and decouple prosody from speaker representations. Despite
aggressive downsampling of the input and having no access to linguistic
information, our model performs comparably to state-of-the-art speech
representations on DAMMP, a new benchmark we introduce for spoken language
understanding. We use minimum description length probing to show that our
representations have selectively learned the subcomponents of non-timbral
prosody, and that the product quantizer naturally disentangles them without
using bottlenecks. We derive an information-theoretic definition of speech
de-identifiability and use it to demonstrate that our prosody representations
are less identifiable than other speech representations.
