  To determine the safety of large language models (LLMs), AI developers must
be able to assess their dangerous capabilities. But simple prompting strategies
often fail to elicit an LLM's full capabilities. One way to elicit capabilities
more robustly is to fine-tune the LLM to complete the task. In this paper, we
investigate the conditions under which fine-tuning-based elicitation suffices
to elicit capabilities. To do this, we introduce password-locked models, LLMs
fine-tuned such that some of their capabilities are deliberately hidden.
Specifically, these LLMs are trained to exhibit these capabilities only when a
password is present in the prompt, and to imitate a much weaker LLM otherwise.
Password-locked models enable a novel method of evaluating capabilities
elicitation methods, by testing whether these password-locked capabilities can
be elicited without using the password. We find that a few high-quality
demonstrations are often sufficient to fully elicit password-locked
capabilities. More surprisingly, fine-tuning can elicit other capabilities that
have been locked using the same password, or even different passwords.
Furthermore, when only evaluations, and not demonstrations, are available,
approaches like reinforcement learning are still often able to elicit
capabilities. Overall, our findings suggest that fine-tuning is an effective
method of eliciting hidden capabilities of current models, but may be
unreliable when high-quality demonstrations are not available, e.g. as may be
the case when models' (hidden) capabilities exceed those of human
demonstrators.
