  Language models as a service (LMaaS) enable users to accomplish tasks without
requiring specialized knowledge, simply by paying a service provider. However,
numerous providers offer massive large language model (LLM) services with
variations in latency, performance, and pricing. Consequently, constructing the
cost-saving LLM services invocation strategy with low-latency and
high-performance responses that meet specific task demands becomes a pressing
challenge. This paper provides a comprehensive overview of the LLM services
invocation methods. Technically, we give a formal definition of the problem of
constructing effective invocation strategy in LMaaS and present the LLM
services invocation framework. The framework classifies existing methods into
four different components, including input abstract, semantic cache, solution
design, and output enhancement, which can be freely combined with each other.
Finally, we emphasize the open challenges that have not yet been well addressed
in this task and shed light on future research.
