  Pre-trained models have achieved excellent performance on the dialogue task.
However, for the continual increase of online chit-chat scenarios, directly
fine-tuning these models for each of the new tasks not only explodes the
capacity of the dialogue system on the embedded devices but also causes
knowledge forgetting on pre-trained models and knowledge interference among
diverse dialogue tasks. In this work, we propose a hierarchical inductive
transfer framework to learn and deploy the dialogue skills continually and
efficiently. First, we introduce the adapter module into pre-trained models for
learning new dialogue tasks. As the only trainable module, it is beneficial for
the dialogue system on the embedded devices to acquire new dialogue skills with
negligible additional parameters. Then, for alleviating knowledge interference
between tasks yet benefiting the regularization between them, we further design
hierarchical inductive transfer that enables new tasks to use general knowledge
in the base adapter without being misled by diverse knowledge in task-specific
adapters. Empirical evaluation and analysis indicate that our framework obtains
comparable performance under deployment-friendly model capacity.
