  Prompt-based models have gathered a lot of attention from researchers due to
their remarkable advancements in the fields of zero-shot and few-shot learning.
Developing an effective prompt template plays a critical role. However, prior
studies have mainly focused on prompt vocabulary searching or embedding
initialization within a predefined template with the prompt position fixed. In
this empirical study, we conduct the most comprehensive analysis to date of
prompt position for diverse Natural Language Processing (NLP) tasks. Our
findings quantify the substantial impact prompt position has on model
performance. We observe that the prompt positions used in prior studies are
often sub-optimal, and this observation is consistent even in widely used
instruction-tuned models. These findings suggest prompt position optimisation
as a valuable research direction to augment prompt engineering methodologies
and prompt position-aware instruction tuning as a potential way to build more
robust models in the future.
