  SUPERB was proposed to evaluate the generalizability of self-supervised
learning (SSL) speech models across various tasks. However, it incurs high
computational costs due to the large datasets and diverse tasks. In this paper,
we introduce MiniSUPERB, a lightweight benchmark that efficiently evaluates SSL
speech models with comparable results to SUPERB but lower computational costs
significantly. We carefully select representative tasks, sample datasets, and
extract model representations offline. Our approach achieves a Spearman's rank
correlation of 0.954 and 0.982 with SUPERB Paper and SUPERB Challenge,
respectively. Additionally, we reduce the computational cost by 97% in terms of
Multiply-ACcumulate operations (MACs). Furthermore, we evaluate SSL speech
models in few-shot scenarios and observe significant variations in their
performance. To our knowledge, this is the first study to examine both the
computational cost of the model itself and the cost of evaluating it on a
benchmark.
