  We discover that many natural-language prompts can be replaced by
corresponding prompts that are unintelligible to humans but that provably
elicit similar behavior in language models. We call these prompts "evil twins"
because they are obfuscated and uninterpretable (evil), but at the same time
mimic the functionality of the original natural-language prompts (twins).
Remarkably, evil twins transfer between models. We find these prompts by
solving a maximum-likelihood problem which has applications of independent
interest.
