  Backdoors are hidden behaviors that are only triggered once an AI system has
been deployed. Bad actors looking to create successful backdoors must design
them to avoid activation during training and evaluation. Since data used in
these stages often only contains information about events that have already
occurred, a component of a simple backdoor trigger could be a model recognizing
data that is in the future relative to when it was trained. Through prompting
experiments and by probing internal activations, we show that current large
language models (LLMs) can distinguish past from future events, with probes on
model activations achieving 90% accuracy. We train models with backdoors
triggered by a temporal distributional shift; they activate when the model is
exposed to news headlines beyond their training cut-off dates. Fine-tuning on
helpful, harmless and honest (HHH) data does not work well for removing simpler
backdoor triggers but is effective on our backdoored models, although this
distinction is smaller for the larger-scale model we tested. We also find that
an activation-steering vector representing a model's internal representation of
the date influences the rate of backdoor activation. We take these results as
initial evidence that, at least for models at the modest scale we test,
standard safety measures are enough to remove these backdoors.
