  In a controllable text generation dataset, there exist unannotated attributes
that could provide irrelevant learning signals to models that use it for
training and thus degrade their performance. We propose focused prefix
tuning(FPT) to mitigate the problem and to enable the control to focus on the
desired attribute. Experimental results show that FPT can achieve better
control accuracy and text fluency than baseline models in single-attribute
control tasks. In multi-attribute control tasks, FPT achieves comparable
control accuracy with the state-of-the-art approach while keeping the
flexibility to control new attributes without retraining existing models.
