  Skill Extraction involves identifying skills and qualifications mentioned in
documents such as job postings and resumes. The task is commonly tackled by
training supervised models using a sequence labeling approach with BIO tags.
However, the reliance on manually annotated data limits the generalizability of
such approaches. Moreover, the common BIO setting limits the ability of the
models to capture complex skill patterns and handle ambiguous mentions. In this
paper, we explore the use of in-context learning to overcome these challenges,
on a benchmark of 6 uniformized skill extraction datasets. Our approach
leverages the few-shot learning capabilities of large language models (LLMs) to
identify and extract skills from sentences. We show that LLMs, despite not
being on par with traditional supervised models in terms of performance, can
better handle syntactically complex skill mentions in skill extraction tasks.
