  A long-running goal of the clinical NLP community is the extraction of
important variables trapped in clinical notes. However, roadblocks have
included dataset shift from the general domain and a lack of public clinical
corpora and annotations. In this work, we show that large language models, such
as InstructGPT, perform well at zero- and few-shot information extraction from
clinical text despite not being trained specifically for the clinical domain.
Whereas text classification and generation performance have already been
studied extensively in such models, here we additionally demonstrate how to
leverage them to tackle a diverse set of NLP tasks which require more
structured outputs, including span identification, token-level sequence
classification, and relation extraction. Further, due to the dearth of
available data to evaluate these systems, we introduce new datasets for
benchmarking few-shot clinical information extraction based on a manual
re-annotation of the CASI dataset for new tasks. On the clinical extraction
tasks we studied, the GPT-3 systems significantly outperform existing zero- and
few-shot baselines.
