  We propose a fully unsupervised method to detect bias in contextualized
embeddings. The method leverages the assortative information latently encoded
by social networks and combines orthogonality regularization, structured
sparsity learning, and graph neural networks to find the embedding subspace
capturing this information. As a concrete example, we focus on the phenomenon
of ideological bias: we introduce the concept of an ideological subspace, show
how it can be found by applying our method to online discussion forums, and
present techniques to probe it. Our experiments suggest that the ideological
subspace encodes abstract evaluative semantics and reflects changes in the
political left-right spectrum during the presidency of Donald Trump.
