  Computer vision applications like automated face detection are used for a
variety of purposes ranging from unlocking smart devices to tracking potential
persons of interest for surveillance. Audits of these applications have
revealed that they tend to be biased against minority groups which result in
unfair and concerning societal and political outcomes. Despite multiple studies
over time, these biases have not been mitigated completely and have in fact
increased for certain tasks like age prediction. While such systems are audited
over benchmark datasets, it becomes necessary to evaluate their robustness for
adversarial inputs. In this work, we perform an extensive adversarial audit on
multiple systems and datasets, making a number of concerning observations -
there has been a drop in accuracy for some tasks on CELEBSET dataset since a
previous audit. While there still exists a bias in accuracy against individuals
from minority groups for multiple datasets, a more worrying observation is that
these biases tend to get exorbitantly pronounced with adversarial inputs toward
the minority group. We conclude with a discussion on the broader societal
impacts in light of these observations and a few suggestions on how to
collectively deal with this issue.
