  Self-supervised language models are very effective at predicting high-level
cortical responses during language comprehension. However, the best current
models of lower-level auditory processing in the human brain rely on either
hand-constructed acoustic filters or representations from supervised audio
neural networks. In this work, we capitalize on the progress of self-supervised
speech representation learning (SSL) to create new state-of-the-art models of
the human auditory system. Compared against acoustic baselines, phonemic
features, and supervised models, representations from the middle layers of
self-supervised models (APC, wav2vec, wav2vec 2.0, and HuBERT) consistently
yield the best prediction performance for fMRI recordings within the auditory
cortex (AC). Brain areas involved in low-level auditory processing exhibit a
preference for earlier SSL model layers, whereas higher-level semantic areas
prefer later layers. We show that these trends are due to the models' ability
to encode information at multiple linguistic levels (acoustic, phonetic, and
lexical) along their representation depth. Overall, these results show that
self-supervised models effectively capture the hierarchy of information
relevant to different stages of speech processing in human cortex.
