  Methods for erasing human-interpretable concepts from neural representations
that assume linearity have been found to be tractable and useful. However, the
impact of this removal on the behavior of downstream classifiers trained on the
modified representations is not fully understood. In this work, we formally
define the notion of log-linear guardedness as the inability of an adversary to
predict the concept directly from the representation, and study its
implications. We show that, in the binary case, under certain assumptions, a
downstream log-linear model cannot recover the erased concept. However, we
demonstrate that a multiclass log-linear model \emph{can} be constructed that
indirectly recovers the concept in some cases, pointing to the inherent
limitations of log-linear guardedness as a downstream bias mitigation
technique. These findings shed light on the theoretical limitations of linear
erasure methods and highlight the need for further research on the connections
between intrinsic and extrinsic bias in neural models.
