  Growing literature has shown that NLP systems may encode social biases;
however, the political bias of summarization models remains relatively unknown.
In this work, we use an entity replacement method to investigate the portrayal
of politicians in automatically generated summaries of news articles. We
develop an entity-based computational framework to assess the sensitivities of
several extractive and abstractive summarizers to the politicians Donald Trump
and Joe Biden. We find consistent differences in these summaries upon entity
replacement, such as reduced emphasis of Trump's presence in the context of the
same article and a more individualistic representation of Trump with respect to
the collective US government (i.e., administration). These summary
dissimilarities are most prominent when the entity is heavily featured in the
source article. Our characterization provides a foundation for future studies
of bias in summarization and for normative discussions on the ideal qualities
of automatic summaries.
