  Large Language Models (LLMs) have revolutionized natural language processing
but can exhibit biases and may generate toxic content. While alignment
techniques like Reinforcement Learning from Human Feedback (RLHF) reduce these
issues, their impact on creativity, defined as syntactic and semantic
diversity, remains unexplored. We investigate the unintended consequences of
RLHF on the creativity of LLMs through three experiments focusing on the
Llama-2 series. Our findings reveal that aligned models exhibit lower entropy
in token predictions, form distinct clusters in the embedding space, and
gravitate towards "attractor states", indicating limited output diversity. Our
findings have significant implications for marketers who rely on LLMs for
creative tasks such as copywriting, ad creation, and customer persona
generation. The trade-off between consistency and creativity in aligned models
should be carefully considered when selecting the appropriate model for a given
application. We also discuss the importance of prompt engineering in harnessing
the creative potential of base models.
