  Large Language Models (LLMs) are being used for a wide variety of tasks.
While they are capable of generating human-like responses, they can also
produce undesirable output including potentially harmful information, racist or
sexist language, and hallucinations. Alignment methods are designed to reduce
such undesirable output, via techniques such as fine-tuning, prompt
engineering, and representation engineering. However, existing methods face
several challenges: some require costly fine-tuning for every alignment task;
some do not adequately remove undesirable concepts, failing alignment; some
remove benign concepts, lowering the linguistic capabilities of LLMs. To
address these issues, we propose Parsimonious Concept Engineering (PaCE), a
novel activation engineering framework for alignment. First, to sufficiently
model the concepts, we construct a large-scale concept dictionary in the
activation space, in which each atom corresponds to a semantic concept. Then,
given any alignment task, we instruct a concept partitioner to efficiently
annotate the concepts as benign or undesirable. Finally, at inference time, we
decompose the LLM activations along the concept dictionary via sparse coding,
to accurately represent the activation as a linear combination of the benign
and undesirable components. By removing the latter ones from the activation, we
reorient the behavior of LLMs towards alignment goals. We conduct experiments
on tasks such as response detoxification, faithfulness enhancement, and
sentiment revising, and show that PaCE achieves state-of-the-art alignment
performance while maintaining linguistic capabilities.
