  Despite recent competitive performance across a range of vision tasks, vision
Transformers still have an issue of heavy computational costs. Recently, vision
prompt learning has provided an economic solution to this problem without
fine-tuning the whole large-scale models. However, the efficiency of existing
models are still far from satisfactory due to insertion of extensive prompts
blocks and trick prompt designs. In this paper, we propose an efficient vision
model named impLicit vIsion prOmpt tuNing (LION), which is motivated by deep
implicit models with stable memory costs for various complex tasks. In
particular, we merely insect two equilibrium implicit layers in two ends of the
pre-trained main backbone with parameters in the backbone frozen. Moreover, we
prune the parameters in these two layers according to lottery hypothesis. The
performance obtained by our LION are promising on a wide range of datasets. In
particular, our LION reduces up to 11.5% of training parameter numbers while
obtaining higher performance compared with the state-of-the-art baseline VPT,
especially under challenging scenes. Furthermore, we find that our proposed
LION had a good generalization performance, making it an easy way to boost
transfer learning in the future.
