  Recent advancements in textless speech-to-speech translation systems have
been driven by the adoption of self-supervised learning techniques. Although
most state-of-the-art systems adopt a similar architecture to transform source
language speech into sequences of discrete representations in the target
language, the criteria for selecting these target speech units remains an open
question. This work explores the selection process through a study of
downstream tasks such as automatic speech recognition, speech synthesis,
speaker recognition, and emotion recognition. Interestingly, our findings
reveal a discrepancy in the optimization of discrete speech units: units that
perform well in resynthesis performance do not necessarily correlate with those
that enhance translation efficacy. This discrepancy underscores the nuanced
complexity of target feature selection and its impact on the overall
performance of speech-to-speech translation systems.
