  Vocabulary transfer is a transfer learning subtask in which language models
fine-tune with the corpus-specific tokenization instead of the default one,
which is being used during pretraining. This usually improves the resulting
performance of the model, and in the paper, we demonstrate that vocabulary
transfer is especially beneficial for medical text processing. Using three
different medical natural language processing datasets, we show vocabulary
transfer to provide up to ten extra percentage points for the downstream
classifier accuracy.
