  Even as pre-trained language encoders such as BERT are shared across many
tasks, the output layers of question answering, text classification, and
regression models are significantly different. Span decoders are frequently
used for question answering, fixed-class, classification layers for text
classification, and similarity-scoring layers for regression tasks, We show
that this distinction is not necessary and that all three can be unified as
span extraction. A unified, span-extraction approach leads to superior or
comparable performance in supplementary supervised pre-trained, low-data, and
multi-task learning experiments on several question answering, text
classification, and regression benchmarks.
