  Training keyphrase generation (KPG) models require a large amount of
annotated data, which can be prohibitively expensive and often limited to
specific domains. In this study, we first demonstrate that large distribution
shifts among different domains severely hinder the transferability of KPG
models. We then propose a three-stage pipeline, which gradually guides KPG
models' learning focus from general syntactical features to domain-related
semantics, in a data-efficient manner. With Domain-general Phrase pre-training,
we pre-train Sequence-to-Sequence models with generic phrase annotations that
are widely available on the web, which enables the models to generate phrases
in a wide range of domains. The resulting model is then applied in the Transfer
Labeling stage to produce domain-specific pseudo keyphrases, which help adapt
models to a new domain. Finally, we fine-tune the model with limited data with
true labels to fully adapt it to the target domain. Our experiment results show
that the proposed process can produce good-quality keyphrases in new domains
and achieve consistent improvements after adaptation with limited in-domain
annotated data. All code and datasets are available at
https://github.com/memray/OpenNMT-kpg-release.
