  The field of adversarial machine learning has experienced a near exponential
growth in the amount of papers being produced since 2018. This massive
information output has yet to be properly processed and categorized. In this
paper, we seek to help alleviate this problem by systematizing the recent
advances in adversarial machine learning black-box attacks since 2019. Our
survey summarizes and categorizes 20 recent black-box attacks. We also present
a new analysis for understanding the attack success rate with respect to the
adversarial model used in each paper. Overall, our paper surveys a wide body of
literature to highlight recent attack developments and organizes them into four
attack categories: score based attacks, decision based attacks, transfer
attacks and non-traditional attacks. Further, we provide a new mathematical
framework to show exactly how attack results can fairly be compared.
