  Many studies have revealed that word embeddings, language models, and models
for specific downstream tasks in NLP are prone to social biases, especially
gender bias. Recently these techniques have been gradually applied to automatic
evaluation metrics for text generation. In the paper, we propose an evaluation
method based on Word Embeddings Association Test (WEAT) and Sentence Embeddings
Association Test (SEAT) to quantify social biases in evaluation metrics and
discover that social biases are also widely present in some model-based
automatic evaluation metrics. Moreover, we construct gender-swapped
meta-evaluation datasets to explore the potential impact of gender bias in
image caption and text summarization tasks. Results show that given
gender-neutral references in the evaluation, model-based evaluation metrics may
show a preference for the male hypothesis, and the performance of them, i.e.
the correlation between evaluation metrics and human judgments, usually has
more significant variation after gender swapping.
