  Large Language Models (LLMs) have become extremely potent instruments with
exceptional capacities for comprehending and producing human-like text in a
wide range of applications. However, the increasing size and complexity of LLMs
present significant challenges in both training and deployment, leading to
substantial computational and storage costs as well as heightened energy
consumption. In this paper, we provide a review of recent advancements and
research directions aimed at addressing these challenges and enhancing the
efficiency of LLM-based systems. We begin by discussing algorithm-level
acceleration techniques focused on optimizing LLM inference speed and resource
utilization. We also explore LLM-hardware co-design strategies with a vision to
improve system efficiency by tailoring hardware architectures to LLM
requirements. Further, we delve into LLM-to-accelerator compilation approaches,
which involve customizing hardware accelerators for efficient LLM deployment.
Finally, as a case study to leverage LLMs for assisting circuit design, we
examine LLM-aided design methodologies for an important task: High-Level
Synthesis (HLS) functional verification, by creating a new dataset that
contains a large number of buggy and bug-free codes, which can be essential for
training LLMs to specialize on HLS verification and debugging. For each aspect
mentioned above, we begin with a detailed background study, followed by the
presentation of several novel solutions proposed to overcome specific
challenges. We then outline future research directions to drive further
advancements. Through these efforts, we aim to pave the way for more efficient
and scalable deployment of LLMs across a diverse range of applications.
