  The growing adoption and deployment of Machine Learning (ML) systems came
with its share of ethical incidents and societal concerns. It also unveiled the
necessity to properly audit these systems in light of ethical principles. For
such a novel type of algorithmic auditing to become standard practice, two main
prerequisites need to be available: A lifecycle model that is tailored towards
transparency and accountability, and a principled risk assessment procedure
that allows the proper scoping of the audit. Aiming to make a pragmatic step
towards a wider adoption of ML auditing, we present a respective procedure that
extends the AI-HLEG guidelines published by the European Commission. Our audit
procedure is based on an ML lifecycle model that explicitly focuses on
documentation, accountability, and quality assurance; and serves as a common
ground for alignment between the auditors and the audited organisation. We
describe two pilots conducted on real-world use cases from two different
organisations and discuss the shortcomings of ML algorithmic auditing as well
as future directions thereof.
