  Pretrained large models (PLMs), such as ChatGPT, have demonstrated remarkable
performance across diverse tasks. However, the significant computational
requirements of PLMs have discouraged most product teams from running or
fine-tuning them. In such cases, to harness the exceptional performance of
PLMs, one must rely on expensive APIs, thereby exacerbating the economic
burden. Despite the overall inferior performance of small models, in specific
distributions, they can achieve comparable or even superior results.
Consequently, some input can be processed exclusively by small models. On the
other hand, certain tasks can be broken down into multiple subtasks, some of
which can be completed without powerful capabilities. Under these
circumstances, small models can handle the simple subtasks, allowing large
models to focus on challenging subtasks, thus improving the performance. We
propose Data Shunt$^+$ (DS$^+$), a general paradigm for collaboration of small
and large models. DS$^+$ not only substantially reduces the cost associated
with querying large models but also effectively improves large models'
performance. For instance, ChatGPT achieves an accuracy of $94.43\%$ on Amazon
Product sentiment analysis, and DS$^+$ achieves an accuracy of $95.64\%$, while
the cost has been reduced to only $31.18\%$. Besides, experiments also prove
that the proposed collaborative-based paradigm can better inject specific task
knowledge into PLMs compared to fine-tuning.
