  Prompt learning with immensely large Casual Language Models (CLMs) has been
shown promising for attribute-controllable text generation (CTG). However,
vanilla prompt tuning tends to imitate training corpus characteristics beyond
the control attributes, resulting in a poor generalization ability. Moreover,
it is less able to capture the relationship between different attributes,
further limiting the control performance. In this paper, we propose a new CTG
approach, namely DisCup, which incorporates the attribute knowledge of
discriminator to optimize the control-prompts, steering a frozen CLM to produce
attribute-specific texts. Specifically, the frozen CLM model, capable of
producing multitudinous texts, is first used to generate the next-token
candidates based on the context, so as to ensure the diversity of tokens to be
predicted. Then, we leverage an attribute-discriminator to select
desired/undesired tokens from those candidates, providing the inter-attribute
knowledge. Finally, we bridge the above two traits by an unlikelihood objective
for prompt-tuning. Extensive experimental results show that DisCup can achieve
a new state-of-the-art control performance while maintaining an efficient and
high-quality text generation, only relying on around 10 virtual tokens.
