  Despite excellent average-case performance of many image classifiers, their
performance can substantially deteriorate on semantically coherent subgroups of
the data that were under-represented in the training data. These systematic
errors can impact both fairness for demographic minority groups as well as
robustness and safety under domain shift. A major challenge is to identify such
subgroups with subpar performance when the subgroups are not annotated and
their occurrence is very rare. We leverage recent advances in text-to-image
models and search in the space of textual descriptions of subgroups ("prompts")
for subgroups where the target model has low performance on the
prompt-conditioned synthesized data. To tackle the exponentially growing number
of subgroups, we employ combinatorial testing. We denote this procedure as
PromptAttack as it can be interpreted as an adversarial attack in a prompt
space. We study subgroup coverage and identifiability with PromptAttack in a
controlled setting and find that it identifies systematic errors with high
accuracy. Thereupon, we apply PromptAttack to ImageNet classifiers and identify
novel systematic errors on rare subgroups.
