  What should regulators of complex algorithms regulate? We propose a model of
oversight over 'black-box' algorithms used in high-stakes applications such as
lending, medical testing, or hiring. In our model, a regulator is limited in
how much she can learn about a black-box model deployed by an agent with
misaligned preferences. The regulator faces two choices: first, whether to
allow for the use of complex algorithms; and second, which key properties of
algorithms to regulate. We show that limiting agents to algorithms that are
simple enough to be fully transparent is inefficient as long as the
misalignment is limited and complex algorithms have sufficiently better
performance than simple ones. Allowing for complex algorithms can improve
welfare, but the gains depend on how the regulator regulates them. Regulation
that focuses on the overall average behavior of algorithms, for example based
on standard explainer tools, will generally be inefficient. Targeted regulation
that focuses on the source of incentive misalignment, e.g., excess false
positives or racial disparities, can provide second-best solutions. We provide
empirical support for our theoretical findings using an application in consumer
lending, where we document that complex models regulated based on
context-specific explanation tools outperform simple, fully transparent models.
This gain from complex models represents a Pareto improvement across our
empirical applications that is preferred both by the lender and from the
perspective of the financial regulator.
