  Skill Extraction (SE) is an important and widely-studied task useful to gain
insights into labor market dynamics. However, there is a lacuna of datasets and
annotation guidelines; available datasets are few and contain crowd-sourced
labels on the span-level or labels from a predefined skill inventory. To
address this gap, we introduce SKILLSPAN, a novel SE dataset consisting of
14.5K sentences and over 12.5K annotated spans. We release its respective
guidelines created over three different sources annotated for hard and soft
skills by domain experts. We introduce a BERT baseline (Devlin et al., 2019).
To improve upon this baseline, we experiment with language models that are
optimized for long spans (Joshi et al., 2020; Beltagy et al., 2020), continuous
pre-training on the job posting domain (Han and Eisenstein, 2019; Gururangan et
al., 2020), and multi-task learning (Caruana, 1997). Our results show that the
domain-adapted models significantly outperform their non-adapted counterparts,
and single-task outperforms multi-task learning.
