  Since the launch of ChatGPT, a powerful AI Chatbot developed by OpenAI, large
language models (LLMs) have made significant advancements in both academia and
industry, bringing about a fundamental engineering paradigm shift in many
areas. While LLMs are powerful, it is also crucial to best use their power
where "prompt'' plays a core role. However, the booming LLMs themselves,
including excellent APIs like ChatGPT, have several inherent limitations: 1)
temporal lag of training data, and 2) the lack of physical capabilities to
perform external actions. Recently, we have observed the trend of utilizing
prompt-based tools to better utilize the power of LLMs for downstream tasks,
but a lack of systematic literature and standardized terminology, partly due to
the rapid evolution of this field. Therefore, in this work, we survey related
prompting tools and promote the concept of the "Prompting Framework" (PF), i.e.
the framework for managing, simplifying, and facilitating interaction with
large language models. We define the lifecycle of the PF as a hierarchical
structure, from bottom to top, namely: Data Level, Base Level, Execute Level,
and Service Level. We also systematically depict the overall landscape of the
emerging PF field and discuss potential future research and challenges. To
continuously track the developments in this area, we maintain a repository at
https://github.com/lxx0628/Prompting-Framework-Survey, which can be a useful
resource sharing platform for both academic and industry in this field.
