  Prompting inputs with natural language task descriptions has emerged as a
popular mechanism to elicit reasonably accurate outputs from large-scale
generative language models with little to no in-context supervision. This also
helps gain insight into how well language models capture the semantics of a
wide range of downstream tasks purely from self-supervised pre-training on
massive corpora of unlabeled text. Such models have naturally also been exposed
to a lot of undesirable content like racist and sexist language and there is
limited work on awareness of models along these dimensions. In this paper, we
define and comprehensively evaluate how well such language models capture the
semantics of four tasks for bias: diagnosis, identification, extraction and
rephrasing. We define three broad classes of task descriptions for these tasks:
statement, question, and completion, with numerous lexical variants within each
class. We study the efficacy of prompting for each task using these classes and
the null task description across several decoding methods and few-shot
examples. Our analyses indicate that language models are capable of performing
these tasks to widely varying degrees across different bias dimensions, such as
gender and political affiliation. We believe our work is an important step
towards unbiased language models by quantifying the limits of current
self-supervision objectives at accomplishing such sociologically challenging
tasks.
