  Instruction-tuned Large Language Models (LLMs) show impressive results in
numerous practical applications, but they lack essential safety features that
are common in other areas of computer science, particularly an explicit
separation of instructions and data. This makes them vulnerable to
manipulations such as indirect prompt injections and generally unsuitable for
safety-critical tasks. Surprisingly, there is currently no established
definition or benchmark to quantify this phenomenon. In this work, we close
this gap by introducing a formal measure for instruction-data separation and an
empirical variant that is calculable from a model's outputs. We also present a
new dataset, SEP, that allows estimating the measure for real-world models. Our
results on various LLMs show that the problem of instruction-data separation is
real: all models fail to achieve high separation, and canonical mitigation
techniques, such as prompt engineering and fine-tuning, either fail to
substantially improve separation or reduce model utility. The source code and
SEP dataset are openly accessible at
https://github.com/egozverev/Shold-It-Be-Executed-Or-Processed.
