  Algorithmic interpretability is necessary to build trust, ensure fairness,
and track accountability. However, there is no existing formal measurement
method for algorithmic interpretability. In this work, we build upon
programming language theory and cognitive load theory to develop a framework
for measuring algorithmic interpretability. The proposed measurement framework
reflects the process of a human learning an algorithm. We show that the
measurement framework and the resulting cognitive complexity score have the
following desirable properties - universality, computability, uniqueness, and
monotonicity. We illustrate the measurement framework through a toy example,
describe the framework and its conceptual underpinnings, and demonstrate the
benefits of the framework, in particular for managers considering tradeoffs
when selecting algorithms.
