  Self-supervised learning (SSL) speech models generate meaningful
representations of given clips and achieve incredible performance across
various downstream tasks. Model extraction attack (MEA) often refers to an
adversary stealing the functionality of the victim model with only query
access. In this work, we study the MEA problem against SSL speech model with a
small number of queries. We propose a two-stage framework to extract the model.
In the first stage, SSL is conducted on the large-scale unlabeled corpus to
pre-train a small speech model. Secondly, we actively sample a small portion of
clips from the unlabeled corpus and query the target model with these clips to
acquire their representations as labels for the small model's second-stage
training. Experiment results show that our sampling methods can effectively
extract the target model without knowing any information about its model
architecture.
