  Gender bias in artificial intelligence (AI) has emerged as a pressing concern
with profound implications for individuals' lives. This paper presents a
comprehensive survey that explores gender bias in Transformer models from a
linguistic perspective. While the existence of gender bias in language models
has been acknowledged in previous studies, there remains a lack of consensus on
how to effectively measure and evaluate this bias. Our survey critically
examines the existing literature on gender bias in Transformers, shedding light
on the diverse methodologies and metrics employed to assess bias. Several
limitations in current approaches to measuring gender bias in Transformers are
identified, encompassing the utilization of incomplete or flawed metrics,
inadequate dataset sizes, and a dearth of standardization in evaluation
methods. Furthermore, our survey delves into the potential ramifications of
gender bias in Transformers for downstream applications, including dialogue
systems and machine translation. We underscore the importance of fostering
equity and fairness in these systems by emphasizing the need for heightened
awareness and accountability in developing and deploying language technologies.
This paper serves as a comprehensive overview of gender bias in Transformer
models, providing novel insights and offering valuable directions for future
research in this critical domain.
