  It has been demonstrated that the art of prompt tuning is highly effective in
efficiently extracting knowledge from pretrained foundation models,
encompassing pretrained language models (PLMs), vision pretrained models, and
vision-language (V-L) models. However, the efficacy of employing fixed soft
prompts with a predetermined position for concatenation with inputs for all
instances, irrespective of their inherent disparities, remains uncertain.
Variables such as the position, length, and representations of prompts across
diverse instances and tasks can substantially influence the performance of
prompt tuning. In this context, we provide a theoretical analysis, which
reveals that optimizing the position of the prompt to encompass the input can
capture additional semantic information that traditional prefix or postfix
prompt tuning methods fail to capture. Building upon our analysis, we present a
unified dynamic prompt (DP) tuning strategy that dynamically determines
different factors of prompts based on specific tasks and instances. To
accomplish this, we employ a lightweight learning network with Gumble-Softmax,
allowing us to learn instance-dependent guidance. Experimental results
underscore the significant performance improvement achieved by dynamic prompt
tuning across a wide range of tasks, including NLP tasks, vision recognition
tasks, and vision-language tasks. Furthermore, we establish the universal
applicability of our approach under full-data, few-shot, and multitask
scenarios. Codes are available at https://github.com/Xianjun-Yang/DPT.
