  Language model alignment has become an important component of AI safety,
allowing safe interactions between humans and language models, by enhancing
desired behaviors and inhibiting undesired ones. It is often done by tuning the
model or inserting preset aligning prompts. Recently, representation
engineering, a method which alters the model's behavior via changing its
representations post-training, was shown to be effective in aligning LLMs (Zou
et al., 2023a). Representation engineering yields gains in alignment oriented
tasks such as resistance to adversarial attacks and reduction of social biases,
but was also shown to cause a decrease in the ability of the model to perform
basic tasks. In this paper we study the tradeoff between the increase in
alignment and decrease in helpfulness of the model. We propose a theoretical
framework which provides bounds for these two quantities, and demonstrate their
relevance empirically. First, we find that under the conditions of our
framework, alignment can be guaranteed with representation engineering, and at
the same time that helpfulness is harmed in the process. Second, we show that
helpfulness is harmed quadratically with the norm of the representation
engineering vector, while the alignment increases linearly with it, indicating
a regime in which it is efficient to use representation engineering. We
validate our findings empirically, and chart the boundaries to the usefulness
of representation engineering for alignment.
