  Transformer models have shown impressive performance on a variety of NLP
tasks. Off-the-shelf, pre-trained models can be fine-tuned for specific NLP
classification tasks, reducing the need for large amounts of additional
training data. However, little research has addressed how much data is required
to accurately fine-tune such pre-trained transformer models, and how much data
is needed for accurate prediction. This paper explores the usability of BERT (a
Transformer model for word embedding) for gender prediction on social media.
Forensic applications include detecting gender obfuscation, e.g. males posing
as females in chat rooms. A Dutch BERT model is fine-tuned on different samples
of a Dutch Twitter dataset labeled for gender, varying in the number of tweets
used per person. The results show that finetuning BERT contributes to good
gender classification performance (80% F1) when finetuned on only 200 tweets
per person. But when using just 20 tweets per person, the performance of our
classifier deteriorates non-steeply (to 70% F1). These results show that even
with relatively small amounts of data, BERT can be fine-tuned to accurately
help predict the gender of Twitter users, and, consequently, that it is
possible to determine gender on the basis of just a low volume of tweets. This
opens up an operational perspective on the swift detection of gender.
