  With the help of simple fine-tuning, one can artificially embed hidden text
into large language models (LLMs). This text is revealed only when triggered by
a specific query to the LLM. Two primary applications are LLM fingerprinting
and steganography. In the context of LLM fingerprinting, a unique text
identifier (fingerprint) is embedded within the model to verify licensing
compliance. In the context of steganography, the LLM serves as a carrier for
hidden messages that can be disclosed through a chosen trigger question.
  Our work demonstrates that embedding hidden text in the LLM via fine-tuning,
though seemingly secure due to the vast number of potential triggers (any
sequence of characters or tokens could serve as a trigger), is susceptible to
extraction through analysis of the LLM's output decoding process. We propose an
extraction attack called Unconditional Token Forcing (UTF). It is premised on
the hypothesis that iteratively feeding each token from the LLM's vocabulary
into the model should reveal output sequences with abnormally high token
probabilities, indicating potential hidden text candidates. We also present a
defense method to hide text in such a way that it is resistant to both UTF and
attacks based on sampling decoding methods, which we named Unconditional Token
Forcing Confusion (UTFC). To the best of our knowledge, there is no attack
method that can extract text hidden with UTFC. UTFC has both benign
applications (improving LLM fingerprinting) and malign applications (using LLMs
to create covert communication channels). Code is available at
github.com/j-hoscilowic/zurek-stegano
