  Detecting disclosures of individuals' employment status on social media can
provide valuable information to match job seekers with suitable vacancies,
offer social protection, or measure labor market flows. However, identifying
such personal disclosures is a challenging task due to their rarity in a sea of
social media content and the variety of linguistic forms used to describe them.
Here, we examine three Active Learning (AL) strategies in real-world settings
of extreme class imbalance, and identify five types of disclosures about
individuals' employment status (e.g. job loss) in three languages using
BERT-based classification models. Our findings show that, even under extreme
imbalance settings, a small number of AL iterations is sufficient to obtain
large and significant gains in precision, recall, and diversity of results
compared to a supervised baseline with the same number of labels. We also find
that no AL strategy consistently outperforms the rest. Qualitative analysis
suggests that AL helps focus the attention mechanism of BERT on core terms and
adjust the boundaries of semantic expansion, highlighting the importance of
interpretable models to provide greater control and visibility into this
dynamic learning process.
