  Prosody plays an important role in characterizing the style of a speaker or
an emotion, but most non-parallel voice or emotion style transfer algorithms do
not convert any prosody information. Two major components of prosody are pitch
and rhythm. Disentangling the prosody information, particularly the rhythm
component, from the speech is challenging because it involves breaking the
synchrony between the input speech and the disentangled speech representation.
As a result, most existing prosody style transfer algorithms would need to rely
on some form of text transcriptions to identify the content information, which
confines their application to high-resource languages only. Recently,
SpeechSplit has made sizeable progress towards unsupervised prosody style
transfer, but it is unable to extract high-level global prosody style in an
unsupervised manner. In this paper, we propose AutoPST, which can disentangle
global prosody style from speech without relying on any text transcriptions.
AutoPST is an Autoencoder-based Prosody Style Transfer framework with a
thorough rhythm removal module guided by the self-expressive representation
learning. Experiments on different style transfer tasks show that AutoPST can
effectively convert prosody that correctly reflects the styles of the target
domains.
