  One of the most concrete measures to take towards meaningful AI
accountability is to consequentially assess and report the systems' performance
and impact. However, the practical nature of the "AI audit" ecosystem is
muddled and imprecise, making it difficult to work through various concepts and
map out the stakeholders involved in the practice. First, we taxonomize current
AI audit practices as completed by regulators, law firms, civil society,
journalism, academia, consulting agencies. Next, we assess the impact of audits
done by stakeholders within each domain. We find that only a subset of AI audit
studies translate to desired accountability outcomes. We thus assess and
isolate practices necessary for effective AI audit results, articulating the
observed connections between AI audit design, methodology and institutional
context on its effectiveness as a meaningful mechanism for accountability.
