  Speech tokenization is the task of representing speech signals as a sequence
of discrete units. Such representations can be later used for various
downstream tasks including automatic speech recognition, text-to-speech, etc.
More relevant to this study, such representation serves as the basis of Speech
Language Models. In this work, we tackle the task of speech tokenization under
the noisy setup and present NAST: Noise Aware Speech Tokenization for Speech
Language Models. NAST is composed of three main components: (i) a predictor;
(ii) a residual encoder; and (iii) a decoder. We evaluate the efficiency of
NAST considering several spoken language modeling tasks and show that NAST is
superior to the evaluated baselines across all setups. Lastly, we analyze NAST
and show its disentanglement properties and robustness to signal variations in
the form of noise, reverberation, pitch-shift, and time-stretch. Code and
pre-trained models are available at https://github.com/ShovalMessica/NAST.
