  Controllable text generation (CTG) seeks to craft texts adhering to specific
attributes, traditionally employing learning-based techniques such as training,
fine-tuning, or prefix-tuning with attribute-specific datasets. These
approaches, while effective, demand extensive computational and data resources.
In contrast, some proposed learning-free alternatives circumvent learning but
often yield inferior results, exemplifying the fundamental machine learning
trade-off between computational expense and model efficacy. To overcome these
limitations, we propose FreeCtrl, a learning-free approach that dynamically
adjusts the weights of selected feedforward neural network (FFN) vectors to
steer the outputs of large language models (LLMs). FreeCtrl hinges on the
principle that the weights of different FFN vectors influence the likelihood of
different tokens appearing in the output. By identifying and adaptively
adjusting the weights of attribute-related FFN vectors, FreeCtrl can control
the output likelihood of attribute keywords in the generated content. Extensive
experiments on single- and multi-attribute control reveal that the
learning-free FreeCtrl outperforms other learning-free and learning-based
methods, successfully resolving the dilemma between learning costs and model
performance.
