  Auditing trained deep learning (DL) models prior to deployment is vital for
preventing unintended consequences. One of the biggest challenges in auditing
is the lack of human-interpretable specifications for the DL models that are
directly useful to the auditor. We address this challenge through a sequence of
semantically-aligned unit tests, where each unit test verifies whether a
predefined specification (e.g., accuracy over 95%) is satisfied with respect to
controlled and semantically aligned variations in the input space (e.g., in
face recognition, the angle relative to the camera). We enable such unit tests
through variations in a semantically-interpretable latent space of a generative
model. Further, we conduct certified training for the DL model through a shared
latent space representation with the generative model. With evaluations on four
different datasets, covering images of chest X-rays, human faces, ImageNet
classes, and towers, we show how AuditAI allows us to obtain controlled
variations for certified training. Thus, our framework, AuditAI, bridges the
gap between semantically-aligned formal verification and scalability. A blog
post accompanying the paper is at this link
https://developer.nvidia.com/blog/nvidia-research-auditing-ai-models-for-verified-deployment-under-semantic-specifications
