  Controllable text generation (CTG) aims to generate text with desired
attributes, and decoding-time-based methods have shown promising performance on
this task. However, in this paper, we identify the phenomenon of Attribute
Collapse for the first time. It causes the fluency of generated text to rapidly
decrease when the control strength exceeds a critical value, rendering the text
completely unusable. This limitation hinders the effectiveness of decoding
methods in achieving high levels of controllability. To address this problem,
we propose a novel lightweight decoding framework named Air-Decoding. Its main
idea is reconstructing the attribute distributions to balance the weights
between attribute words and non-attribute words to generate more fluent text.
Specifically, we train prefixes by prefix-tuning to obtain attribute
distributions. Then we design a novel attribute distribution reconstruction
method to balance the obtained distributions and use the reconstructed
distributions to guide language models for generation, effectively avoiding the
issue of Attribute Collapse. Experiments on multiple CTG tasks prove that our
method achieves a new state-of-the-art control performance.
