  Prompt learning has achieved great success in efficiently exploiting
large-scale pre-trained models in natural language processing (NLP). It
reformulates the downstream tasks as the generative pre-training ones to
achieve consistency, thus improving the performance stably. However, when
transferring it to the vision area, current visual prompt learning methods are
almost designed on discriminative pre-trained models, and there is also a lack
of careful design to unify the forms of pre-training and downstream tasks. To
explore prompt learning on the generative pre-trained visual model, as well as
keeping the task consistency, we propose Visual Prompt learning as masked
visual Token Modeling (VPTM) to transform the downstream visual classification
into the pre-trained masked visual token prediction. In addition, we develop
the prototypical verbalizer for mapping the predicted visual token with
implicit semantics to explicit downstream labels. To our best knowledge, VPTM
is the first visual prompt method on the generative pre-trained visual model,
which achieves consistency between pre-training and downstream visual
classification by task reformulation. Experiments show that VPTM outperforms
other visual prompt methods and achieves excellent efficiency. Moreover, the
task consistency of VPTM contributes to the robustness against prompt location,
prompt length and prototype dimension, and could be deployed uniformly.
