  The literature has witnessed the success of leveraging Pre-trained Language
Models (PLMs) and Transfer Learning (TL) algorithms to a wide range of Natural
Language Processing (NLP) applications, yet it is not easy to build an
easy-to-use and scalable TL toolkit for this purpose. To bridge this gap, the
EasyTransfer platform is designed to develop deep TL algorithms for NLP
applications. EasyTransfer is backended with a high-performance and scalable
engine for efficient training and inference, and also integrates comprehensive
deep TL algorithms, to make the development of industrial-scale TL applications
easier. In EasyTransfer, the built-in data and model parallelism strategies,
combined with AI compiler optimization, show to be 4.0x faster than the
community version of distributed training. EasyTransfer supports various NLP
models in the ModelZoo, including mainstream PLMs and multi-modality models. It
also features various in-house developed TL algorithms, together with the
AppZoo for NLP applications. The toolkit is convenient for users to quickly
start model training, evaluation, and online deployment. EasyTransfer is
currently deployed at Alibaba to support a variety of business scenarios,
including item recommendation, personalized search, conversational question
answering, etc. Extensive experiments on real-world datasets and online
applications show that EasyTransfer is suitable for online production with
cutting-edge performance for various applications. The source code of
EasyTransfer is released at Github (https://github.com/alibaba/EasyTransfer).
