  Machine learning models that incorporate concept learning as an intermediate
step in their decision making process can match the performance of black-box
predictive models while retaining the ability to explain outcomes in human
understandable terms. However, we demonstrate that the concept representations
learned by these models encode information beyond the pre-defined concepts, and
that natural mitigation strategies do not fully work, rendering the
interpretation of the downstream prediction misleading. We describe the
mechanism underlying the information leakage and suggest recourse for
mitigating its effects.
