  Model extraction (ME) attacks represent one major threat to
Machine-Learning-as-a-Service (MLaaS) platforms by ``stealing'' the
functionality of confidential machine-learning models through querying
black-box APIs. Over seven years have passed since ME attacks were first
conceptualized in the seminal work. During this period, substantial advances
have been made in both ME attacks and MLaaS platforms, raising the intriguing
question: How has the vulnerability of MLaaS platforms to ME attacks been
evolving? In this work, we conduct an in-depth study to answer this critical
question. Specifically, we characterize the vulnerability of current,
mainstream MLaaS platforms to ME attacks from multiple perspectives including
attack strategies, learning techniques, surrogate-model design, and benchmark
tasks. Many of our findings challenge previously reported results, suggesting
emerging patterns of ME vulnerability. Further, by analyzing the vulnerability
of the same MLaaS platforms using historical datasets from the past four years,
we retrospectively characterize the evolution of ME vulnerability over time,
leading to a set of interesting findings. Finally, we make suggestions about
improving the current practice of MLaaS in terms of attack robustness. Our
study sheds light on the current state of ME vulnerability in the wild and
points to several promising directions for future research.
