  Despite the evolution of language models, they continue to portray harmful
societal biases and stereotypes inadvertently learned from training data. These
inherent biases often result in detrimental effects in various applications.
Counterfactual Data Augmentation (CDA), which seeks to balance demographic
attributes in training data, has been a widely adopted approach to mitigate
bias in natural language processing. However, many existing CDA approaches rely
on word substitution techniques using manually compiled word-pair dictionaries.
These techniques often lead to out-of-context substitutions, resulting in
potential quality issues. The advancement of model-based techniques, on the
other hand, has been challenged by the need for parallel training data. Works
in this area resort to manually generated parallel data that are expensive to
collect and are consequently limited in scale. This paper proposes FairFlow, an
automated approach to generating parallel data for training counterfactual text
generator models that limits the need for human intervention. Furthermore, we
show that FairFlow significantly overcomes the limitations of dictionary-based
word-substitution approaches whilst maintaining good performance.
