  The rapid proliferation of open-source language models significantly
increases the risks of downstream backdoor attacks. These backdoors can
introduce dangerous behaviours during model deployment and can evade detection
by conventional cybersecurity monitoring systems. In this paper, we introduce a
novel class of backdoors in autoregressive transformer models, that, in
contrast to prior art, are unelicitable in nature. Unelicitability prevents the
defender from triggering the backdoor, making it impossible to evaluate or
detect ahead of deployment even if given full white-box access and using
automated techniques, such as red-teaming or certain formal verification
methods. We show that our novel construction is not only unelicitable thanks to
using cryptographic techniques, but also has favourable robustness properties.
We confirm these properties in empirical investigations, and provide evidence
that our backdoors can withstand state-of-the-art mitigation strategies.
Additionally, we expand on previous work by showing that our universal
backdoors, while not completely undetectable in white-box settings, can be
harder to detect than some existing designs. By demonstrating the feasibility
of seamlessly integrating backdoors into transformer models, this paper
fundamentally questions the efficacy of pre-deployment detection strategies.
This offers new insights into the offence-defence balance in AI safety and
security.
