  Summarization is an important application of large language models (LLMs).
Most previous evaluation of summarization models has focused on their content
selection, faithfulness, grammaticality and coherence. However, it is well
known that LLMs can reproduce and reinforce harmful social biases. This raises
the question: Do biases affect model outputs in a constrained setting like
summarization? To help answer this question, we first motivate and introduce a
number of definitions for biased behaviours in summarization models, along with
practical operationalizations. Since we find that biases inherent to input
documents can confound bias analysis in summaries, we propose a method to
generate input documents with carefully controlled demographic attributes. This
allows us to study summarizer behavior in a controlled setting, while still
working with realistic input documents. We measure gender bias in English
summaries generated by both purpose-built summarization models and general
purpose chat models as a case study. We find content selection in single
document summarization to be largely unaffected by gender bias, while
hallucinations exhibit evidence of bias. To demonstrate the generality of our
approach, we additionally investigate racial bias, including intersectional
settings.
